Difference between revisions of "BASH script to run bittorrent as a daemon"

From Theory.org Wiki
Jump to: navigation, search
(PhpWikiMigration)
 
m (Spam deleted)
 
(25 intermediate revisions by 14 users not shown)
Line 1: Line 1:
 
By using the btlaunchmany.py executable you can do this:
 
By using the btlaunchmany.py executable you can do this:
  I created a '''torrent directory'' which only contains two other directories: '''active'' and '''standby'' I then download all '''.torrent file into the ''standby''' directory and when I want to start fetching the file I simply move the file over to the ''active* directory.
 
  
I start the [[BitTorrent]] executable as a background task using this script:
+
I created a '''torrent directory''' which only contains two other directories: '''active''' and '''standby''' I then download all .torrent file into the '''standby''' directory and when I want to start fetching the file I simply move the file over to the active directory.
 +
 
 +
I start the [[BitTorrent]] executable as a background task using this script:
 
<pre>
 
<pre>
 
#!/bin/sh
 
#!/bin/sh
Line 10: Line 11:
 
</pre>
 
</pre>
  
It will start the [[BitTorrent]] and scans the ''active'' directory for all *.torrent references and will then download the files in the same directory.
+
It will start the [[BitTorrent]] and scans the ''active'' directory for all *.torrent references and will then download the files in the same directory.  Fortunately the btlaunchmany.py is scanning the ''active'' directory frequently, so by simply moving a *.torrent file into that directory it will be recognized and a new download thread will be created.
  Fortunately the btlaunchmany.py is scanning the ''active'' directory frequently, so by simply moving a *.torrent file into that directory it will be recognized and a new download thread will be created.
 
  
----
+
To avoid this process exiting when you log out, you can either
  
Err.. I see a background process, but won't this be killed when you log out?
+
* use ''nohup''
  
Answer: with nohup:
 
 
<code>nohup btlaunchmany.py torrent/active/ > torrent.log &</code>
 
<code>nohup btlaunchmany.py torrent/active/ > torrent.log &</code>
  
Another answer:  run it inside a program called <code>screen</code>  when you come back type <code>screen -r</code> to reconnect to it
+
* double background it
 
 
---
 
 
 
how do i get this thing to run on RH9?
 
 
 
 
 
---
 
 
 
 
 
 
 
I actually got this running fine on RH9, but it does not seem to pick up new torrents as described above....
 
 
 
---
 
  
 
Probably a better way is to double background it. This is a "better" way to disconnect a background script from a controlling (login) terminal:
 
 
<pre>
 
<pre>
 
 
     (./btlaunchmany.py torrent/active/ > torrent.log 2>&1 &) &
 
     (./btlaunchmany.py torrent/active/ > torrent.log 2>&1 &) &
 
 
</pre>
 
</pre>
  
 
This will display a message something like this:
 
This will display a message something like this:
 
<pre>
 
<pre>
 +
    [1] + Done                ( ./btlaunchmany.py torrent/active/ > torrent.l
 +
</pre>
 +
 +
That's OK. If you do a 'ps -x' you will see your btlauncher running in the background. No nohup required. The '2>&1' redirects stderr to stdout which is redirected to torrent.log. This makes it totally silent. Then it's put into the background with '&' and it's backgrounded twice with another '&'. What this does is totally disconnect the process from your terminal. It may look weird, but this is the UNIX idiom that says "not only do I  want to run this process asynchronously, I also want the parent process to be the INIT process instead of my terminal". Now, if you actually want to kill the process you can send it a HUP signal 'kill -hup {PID}'.
 +
 +
* use the ''screen'' utility
 +
 +
<code>screen</code> 
 +
<code>btlaunchmany.py torrent/active/ > torrent.log &</code>
 +
 +
you may exit your terminal session, and when you relogin you can type
  
    [1] + Done                ( ./btlaunchmany.py torrent/active/ > torrent.l
+
<code>screen -r</code>  
  
</pre>
+
to reconnect to it.
  
That's OK. If you do a 'ps -x' you will see your btlauncher running in the background.
+
RH9 users can run this script by  may not pick up new torrents as described above.
No nohup required. The '2>&1' redirects stderr to stdout which is redirected to torrent.log. This makes it totally silent.
 
Then it's put into the background with '&' and it's backgrounded twice with another '&'. What this does is totally disconnect the process from your terminal. It may look weird, but this is the UNIX idiom that says "not only do I  want to run this process asynchronously, I also want the parent process to be the INIT process instead of my terminal".
 
Now, if you actually want to kill the process you can send it a HUP signal 'kill -hup {PID}'.
 
 
----
 
----
 +
 
I use this one as /etc/init.d/bittorrent on debian:
 
I use this one as /etc/init.d/bittorrent on debian:
 
<pre>
 
<pre>
Line 227: Line 218:
 
# see LICENSE.txt for license information
 
# see LICENSE.txt for license information
  
from [[BitTorrent]].download import download
+
from BitTorrent.download import download
 
from threading import Thread, Event, Lock
 
from threading import Thread, Event, Lock
 
from os import listdir, rename
 
from os import listdir, rename
Line 239: Line 230:
 
print "btlaunchmany.bittorrent is RUNNING"
 
print "btlaunchmany.bittorrent is RUNNING"
  
LOG = "/home/bittorrent/public_html/log.txt";
+
LOG = "/home/bittorrent/public_html/log.txt"
FILE = open(LOG,"w+");
+
FILE = open(LOG,"w+")
old_out = sys.stdout;
+
old_out = sys.stdout
old_err = sys.stderr;
+
old_err = sys.stderr
sys.stdout <code> sys.stderr </code> FILE;
+
sys.stdout = sys.stderr = FILE
  
COMPLETED = "/home/bittorrent/completed/";
+
COMPLETED = "/home/bittorrent/completed/"
  
 
def cleanup():
 
def cleanup():
        global sys, old''out, old''err, FILE;
+
         FILE.close()
         FILE.close();
+
         sys.stdout = old_out
         sys.stdout = old_out;
+
sys.stderr = old_err
sys.stderr = old_err;
 
  
  
 
def sendMail(filename,status):
 
def sendMail(filename,status):
global sys
+
short_filename = basename(filename)
short_filename = basename(filename);
 
 
print "SENDING NOTIFICATION ABOUT %s %s" % (status, short_filename)
 
print "SENDING NOTIFICATION ABOUT %s %s" % (status, short_filename)
 
sys.stdout.flush()
 
sys.stdout.flush()
 
sys.stderr.flus()
 
sys.stderr.flus()
 
server = smtplib.SMTP("localhost")
 
server = smtplib.SMTP("localhost")
server.sendmail("bittorrent@yuriy.org","yuriy@localhost", "Subject: bittorent:%s
+
server.sendmail("bittorrent@yuriy.org","yuriy@localhost", "Subject: bittorent:%s %s has %s download!" % (short_filename,short_filename,status))
 
 
%s has %s download!" % (short''filename,short''filename,status))
 
 
server.quit()
 
server.quit()
  
 
def cleanTorrent(filename):
 
def cleanTorrent(filename):
global sys
+
short_filename = basename(filename)
short_filename = basename(filename);
 
 
print "REMOVING TORRENT %s" % short_filename
 
print "REMOVING TORRENT %s" % short_filename
 
sys.stdout.flush()
 
sys.stdout.flush()
Line 287: Line 273:
 
     return '%d:%02d:%02d' % (h, m, s)
 
     return '%d:%02d:%02d' % (h, m, s)
  
def fmtsize(n, baseunit <code> 0, padded </code> 1):
+
def fmtsize(n, baseunit = 0, padded = 1):
     unit = [[' B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']]
+
     unit = [' B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']
 
     i = baseunit
 
     i = baseunit
 
     while i + 1 < len(unit) and n >= 999:
 
     while i + 1 < len(unit) and n >= 999:
Line 300: Line 286:
 
             size = ' '
 
             size = ' '
 
     if i != 0:
 
     if i != 0:
         size += '%.1f %s' % (n, unit[[i]])
+
         size += '%.1f %s' % (n, unit[i])
 +
    elif padded:
 +
        size += '%.0f  %s' % (n, unit[i])
 
     else:
 
     else:
         if padded:
+
         size += '%.0f %s' % (n, unit[i])
            size += '%.0f   %s' % (n, unit[[i]])
 
        else:
 
            size += '%.0f %s' % (n, unit[[i]])
 
 
     return size
 
     return size
  
  
def dummy('''args, ''*kwargs):
+
def dummy(*args, **kwargs):
 
     pass
 
     pass
  
Line 315: Line 300:
 
ext = '.torrent'
 
ext = '.torrent'
 
print 'btlaunchmany starting..'
 
print 'btlaunchmany starting..'
print '...logging to ' + LOG;
+
print '...logging to ' + LOG
 
filecheck = Lock()
 
filecheck = Lock()
  
 
def dropdir_mainloop(d, params):
 
def dropdir_mainloop(d, params):
 
     deadfiles = []
 
     deadfiles = []
    global threads, status, sys
+
     while True:
     while 1:
 
 
         files = listdir(d)
 
         files = listdir(d)
 
         # new files
 
         # new files
 
         for file in files:
 
         for file in files:
             if file[[-len(ext):]] == ext:
+
             if file.endswith(ext) and \:
                if file not in threads.keys() + deadfiles:
+
              file not in threads.keys() + deadfiles:
                    threads[[file]] = {'kill': Event(), 'try': 1}
+
                threads[file] = {'kill': Event(), 'try': 1}
                    print 'New torrent: %s' % file
+
                print 'New torrent: %s' % file
                    sys.stdout.flush()
+
                sys.stdout.flush()
    sys.stderr.flush()
+
sys.stderr.flush()
                    threads[[file]][['thread']] <code> Thread(target </code> [[StatusUpdater]](join(d, file), params, file).download, name = file)
+
                threads[file]['thread'] = Thread(target = StatusUpdater(join(d, file), params, file).download, name = file)
                    threads[[file]][['thread']].start()
+
                threads[file]['thread'].start()
 
         # files with multiple tries
 
         # files with multiple tries
 
         for file, threadinfo in threads.items():
 
         for file, threadinfo in threads.items():
 
             if threadinfo.get('timeout') == 0:
 
             if threadinfo.get('timeout') == 0:
 
                 # Zero seconds left, try and start the thing again.
 
                 # Zero seconds left, try and start the thing again.
                 threadinfo[['try']] = threadinfo[['try']] + 1
+
                 threadinfo['try'] += 1
                 threadinfo[['thread']] <code> Thread(target </code> [[StatusUpdater]](join(d, file), params, file).download, name = file)
+
                 threadinfo['thread'] = Thread(target = StatusUpdater(join(d, file), params, file).download, name = file)
                 threadinfo[['thread']].start()
+
                 threadinfo['thread'].start()
                 threadinfo[['timeout']] = -1
+
                 threadinfo['timeout'] = -1
 
             elif threadinfo.get('timeout') > 0:
 
             elif threadinfo.get('timeout') > 0:
 
                 # Decrement our counter by 1
 
                 # Decrement our counter by 1
                 threadinfo[['timeout']] = threadinfo[['timeout']] - 1
+
                 threadinfo['timeout'] -= 1
             elif not threadinfo[['thread']].isAlive():
+
             elif not threadinfo['thread'].isAlive():
 
                 # died without permission
 
                 # died without permission
 
                 # if it was checking the file, it isn't anymore.
 
                 # if it was checking the file, it isn't anymore.
                 if threadinfo.get('checking', None):
+
                 if threadinfo.get('checking'):
 
                     filecheck.release()
 
                     filecheck.release()
 
                 if threadinfo.get('try') == 6:
 
                 if threadinfo.get('try') == 6:
Line 355: Line 339:
 
                     sys.stdout.flush()
 
                     sys.stdout.flush()
 
                     sys.stderr.flush()
 
                     sys.stderr.flush()
    del threads[[file]]
+
    del threads[file]
 
                 else:
 
                 else:
                     del threadinfo[['thread']]
+
                     del threadinfo['thread']
                     threadinfo[['timeout']] = 10
+
                     threadinfo['timeout'] = 10
             # dealing with files that dissapear
+
             # dealing with files that disappear
 
             if file not in files:
 
             if file not in files:
                 print 'Torrent file dissapeared, killing %s' % file
+
                 print 'Torrent file disappeared, killing %s' % file
 
                 stdout.flush()
 
                 stdout.flush()
 
                 if threadinfo.get('timeout', -1) == -1:
 
                 if threadinfo.get('timeout', -1) == -1:
                     threadinfo[['kill']].set()
+
                     threadinfo['kill'].set()
                     threadinfo[['thread']].join()
+
                     threadinfo['thread'].join()
 
                 # if this thread was filechecking, open it up
 
                 # if this thread was filechecking, open it up
                 if threadinfo.get('checking', None):
+
                 if threadinfo.get('checking'):
 
                     filecheck.release()
 
                     filecheck.release()
                 del threads[[file]]
+
                 del threads[file]
 
         for file in deadfiles:
 
         for file in deadfiles:
 
             # if the file dissapears, remove it from our dead list
 
             # if the file dissapears, remove it from our dead list
Line 378: Line 362:
 
def display_thread(displaykiller):
 
def display_thread(displaykiller):
 
     interval = 1.0
 
     interval = 1.0
     global threads, status, os, sys
+
     global status
     while 1:
+
     while True:
 
         # display file info
 
         # display file info
         if (displaykiller.isSet()):
+
         if displaykiller.isSet():
 
             break
 
             break
 
         totalup = 0
 
         totalup = 0
Line 392: Line 376:
 
             uprate = threadinfo.get('uprate', 0)
 
             uprate = threadinfo.get('uprate', 0)
 
             downrate = threadinfo.get('downrate', 0)
 
             downrate = threadinfo.get('downrate', 0)
             uptxt <code> fmtsize(uprate, padded </code> 0)
+
             uptxt = fmtsize(uprate, padded = 0)
             downtxt <code> fmtsize(downrate, padded </code> 0)
+
             downtxt = fmtsize(downrate, padded = 0)
 
             uptotal = threadinfo.get('uptotal', 0.0)
 
             uptotal = threadinfo.get('uptotal', 0.0)
 
             downtotal = threadinfo.get('downtotal', 0.0)
 
             downtotal = threadinfo.get('downtotal', 0.0)
             uptotaltxt <code> fmtsize(uptotal, baseunit </code> 2, padded = 0)
+
             uptotaltxt = fmtsize(uptotal, baseunit = 2, padded = 0)
             downtotaltxt <code> fmtsize(downtotal, baseunit </code> 2, padded = 0)
+
             downtotaltxt = fmtsize(downtotal, baseunit = 2, padded = 0)
 
             filename = threadinfo.get('savefile', file)
 
             filename = threadinfo.get('savefile', file)
 
             if threadinfo.get('timeout', 0) > 0:
 
             if threadinfo.get('timeout', 0) > 0:
Line 405: Line 389:
 
             else:
 
             else:
 
                 status = threadinfo.get('status','')
 
                 status = threadinfo.get('status','')
                 print '%s: Spd: %s/s:%s/s Tot: %s:%s [[%s]]' % (basename(filename), uptxt, downtxt, uptotaltxt, downtotaltxt, status)
+
                 print '%s: Spd: %s/s:%s/s Tot: %s:%s [%s]' % (basename(filename), uptxt, downtxt, uptotaltxt, downtotaltxt, status)
 
if status == 'complete' and isfile(filename):
 
if status == 'complete' and isfile(filename):
 
sendMail(filename,status)
 
sendMail(filename,status)
Line 414: Line 398:
 
             totaldowntotal += downtotal
 
             totaldowntotal += downtotal
 
         # display totals line
 
         # display totals line
         totaluptxt <code> fmtsize(totalup, padded </code> 0)
+
         totaluptxt = fmtsize(totalup, padded = 0)
         totaldowntxt <code> fmtsize(totaldown, padded </code> 0)
+
         totaldowntxt = fmtsize(totaldown, padded = 0)
         totaluptotaltxt <code> fmtsize(totaluptotal, baseunit </code> 2, padded = 0)
+
         totaluptotaltxt = fmtsize(totaluptotal, baseunit = 2, padded = 0)
         totaldowntotaltxt <code> fmtsize(totaldowntotal, baseunit </code> 2, padded = 0)
+
         totaldowntotaltxt = fmtsize(totaldowntotal, baseunit = 2, padded = 0)
 
         print 'All: Spd: %s/s:%s/s Tot: %s:%s' % (totaluptxt, totaldowntxt, totaluptotaltxt, totaldowntotaltxt)
 
         print 'All: Spd: %s/s:%s/s Tot: %s:%s' % (totaluptxt, totaldowntxt, totaluptotaltxt, totaldowntotaltxt)
 
         print
 
         print
Line 424: Line 408:
 
sleep(interval)
 
sleep(interval)
  
class [[StatusUpdater]]:
+
class StatusUpdater:
     def _''init''_(self, file, params, name):
+
     def __init__(self, file, params, name):
 
         self.file = file
 
         self.file = file
 
         self.params = params
 
         self.params = params
 
         self.name = name
 
         self.name = name
         self.myinfo = threads[[name]]
+
         self.myinfo = threads[name]
 
         self.done = 0
 
         self.done = 0
 
         self.checking = 0
 
         self.checking = 0
 
         self.activity = 'starting'
 
         self.activity = 'starting'
 
         self.display()
 
         self.display()
         self.myinfo[['errors']] = []
+
         self.myinfo['errors'] = []
  
 
     def download(self):
 
     def download(self):
global sys
+
         download(self.params + ['--responsefile', self.file], self.choose, self.display, self.finished, self.err, self.myinfo['kill'], 80)
         download(self.params + [['--responsefile', self.file]], self.choose, self.display, self.finished, self.err, self.myinfo[['kill']], 80)
 
 
         print 'Torrent %s stopped' % self.file
 
         print 'Torrent %s stopped' % self.file
 
         sys.stdout.flush()
 
         sys.stdout.flush()
Line 444: Line 427:
 
     def finished(self):
 
     def finished(self):
 
         self.done = 1
 
         self.done = 1
         self.myinfo[['done']] = 1
+
         self.myinfo['done'] = 1
 
         self.activity = 'complete'
 
         self.activity = 'complete'
 
         self.display({'fractionDone' : 1, 'downRate' : 0})
 
         self.display({'fractionDone' : 1, 'downRate' : 0})
  
 
     def err(self, msg):
 
     def err(self, msg):
         self.myinfo[['errors']].append(msg)
+
         self.myinfo['errors'].append(msg)
 
         self.display()
 
         self.display()
  
Line 457: Line 440:
  
 
     def choose(self, default, size, saveas, dir):
 
     def choose(self, default, size, saveas, dir):
        global filecheck
+
         self.myinfo['downfile'] = default
         self.myinfo[['downfile']] = default
+
         self.myinfo['filesize'] = fmtsize(size)
         self.myinfo[['filesize']] = fmtsize(size)
 
 
         if saveas == '':
 
         if saveas == '':
 
             saveas = default
 
             saveas = default
 
         # it asks me where I want to save it before checking the file..
 
         # it asks me where I want to save it before checking the file..
         if exists(self.file[[:-len(ext)]]) and (getsize(self.file[[:-len(ext)]]) > 0):
+
         if exists(self.file[:-len(ext)]) and getsize(self.file[:-len(ext)]) > 0:
 
             # file will get checked
 
             # file will get checked
             while (not filecheck.acquire(0) and not self.myinfo[['kill']].isSet()):
+
             while not filecheck.acquire(0) and not self.myinfo['kill'].isSet():
                 self.myinfo[['status']] = 'disk wait'
+
                 self.myinfo['status'] = 'disk wait'
 
                 sleep(0.1)
 
                 sleep(0.1)
             if not self.myinfo[['kill']].isSet():
+
             if not self.myinfo['kill'].isSet():
 
                 self.checking = 1
 
                 self.checking = 1
                 self.myinfo[['checking']] = 1
+
                 self.myinfo['checking'] = 1
         self.myinfo[['savefile']] = self.file[[:-len(ext)]]
+
         self.myinfo['savefile'] = self.file[:-len(ext)]
         return self.file[[:-len(ext)]]
+
         return self.file[:-len(ext)]
  
 
     def display(self, dict = {}):
 
     def display(self, dict = {}):
         fractionDone = dict.get('fractionDone', None)
+
         fractionDone = dict.get('fractionDone')
         timeEst = dict.get('timeEst', None)
+
         timeEst = dict.get('timeEst')
         activity = dict.get('activity', None)
+
         activity = dict.get('activity')
        global status
 
 
         if activity is not None and not self.done:
 
         if activity is not None and not self.done:
 
             if activity == 'checking existing file':
 
             if activity == 'checking existing file':
Line 489: Line 470:
 
             self.activity = fmttime(timeEst)
 
             self.activity = fmttime(timeEst)
 
         if fractionDone is not None:
 
         if fractionDone is not None:
             self.myinfo[['status']] = '%s %.0f%%' % (self.activity, fractionDone * 100)
+
             self.myinfo['status'] = '%s %.0f%%' % (self.activity, fractionDone * 100)
 
         else:
 
         else:
             self.myinfo[['status']] = self.activity
+
             self.myinfo['status'] = self.activity
 
         if self.activity != 'checking existing file' and self.checking:
 
         if self.activity != 'checking existing file' and self.checking:
 
             # we finished checking our files.
 
             # we finished checking our files.
 
             filecheck.release()
 
             filecheck.release()
 
             self.checking = 0
 
             self.checking = 0
             self.myinfo[['checking']] = 0
+
             self.myinfo['checking'] = 0
         if dict.has_key('upRate'):
+
         if 'upRate' in dict:
             self.myinfo[['uprate']] = dict[['upRate']]
+
             self.myinfo['uprate'] = dict['upRate']
         if dict.has_key('downRate'):
+
         if 'downRate' in dict:
             self.myinfo[['downrate']] = dict[['downRate']]
+
             self.myinfo['downrate'] = dict['downRate']
         if dict.has_key('upTotal'):
+
         if 'upTotal' in dict:
             self.myinfo[['uptotal']] = dict[['upTotal']]
+
             self.myinfo['uptotal'] = dict['upTotal']
         if dict.has_key('downTotal'):
+
         if 'downTotal' in dict:
             self.myinfo[['downtotal']] = dict[['downTotal']]
+
             self.myinfo['downtotal'] = dict['downTotal']
  
if _''name'''' == '''''main''_':
+
if __name__ == '__main__':
     if (len(argv) < 2):
+
     if len(argv) < 2:
 
         print """Usage: btlaunchmany.py <directory> <global options>
 
         print """Usage: btlaunchmany.py <directory> <global options>
 
   <directory> - directory to look for .torrent files (non-recursive)
 
   <directory> - directory to look for .torrent files (non-recursive)
Line 515: Line 496:
 
     try:
 
     try:
 
         displaykiller = Event()
 
         displaykiller = Event()
         displaythread <code> Thread(target </code> display_thread, name <code> 'display', args </code> [[displaykiller]])
+
         displaythread = Thread(target = display_thread, name = 'display', args = [displaykiller])
 
         displaythread.start()
 
         displaythread.start()
         dropdir_mainloop(argv[[1]], argv[[2:]])
+
         dropdir_mainloop(argv[1], argv[2:])
     except [[KeyboardInterrupt]]:
+
     except KeyboardInterrupt:
 
         print '^C caught! Killing torrents..'
 
         print '^C caught! Killing torrents..'
 
         for file, threadinfo in threads.items():
 
         for file, threadinfo in threads.items():
 
             status = 'Killing torrent %s' % file
 
             status = 'Killing torrent %s' % file
             threadinfo[['kill']].set()
+
             threadinfo['kill'].set()
             threadinfo[['thread']].join()
+
             threadinfo['thread'].join()
             del threads[[file]]
+
             del threads[file]
 
         displaykiller.set()
 
         displaykiller.set()
 
         displaythread.join()
 
         displaythread.join()
cleanup();
+
cleanup()
 
     except:
 
     except:
 
         traceback.print_exc()
 
         traceback.print_exc()
Line 547: Line 528:
 
FILESDIR=/home/bittorrent/active
 
FILESDIR=/home/bittorrent/active
 
TORRENTSDIR=/var/www/torrents
 
TORRENTSDIR=/var/www/torrents
SERVER=http://${your''sever}:${your''port}
+
SERVER=http://${your_sever}:${your_port}
  
 
OPTIONS="--dfile ./$DFILE --port $PORT"
 
OPTIONS="--dfile ./$DFILE --port $PORT"

Latest revision as of 06:58, 7 August 2012

By using the btlaunchmany.py executable you can do this:

I created a torrent directory which only contains two other directories: active and standby I then download all .torrent file into the standby directory and when I want to start fetching the file I simply move the file over to the active directory.

I start the BitTorrent executable as a background task using this script:

#!/bin/sh
cd
nohup btlaunchmany.py torrent/active/ > torrent.log &
tail -f torrent.log

It will start the BitTorrent and scans the active directory for all *.torrent references and will then download the files in the same directory. Fortunately the btlaunchmany.py is scanning the active directory frequently, so by simply moving a *.torrent file into that directory it will be recognized and a new download thread will be created.

To avoid this process exiting when you log out, you can either

  • use nohup

nohup btlaunchmany.py torrent/active/ > torrent.log &

  • double background it
    (./btlaunchmany.py torrent/active/ > torrent.log 2>&1 &) &

This will display a message something like this:

    [1] + Done                 ( ./btlaunchmany.py torrent/active/ > torrent.l

That's OK. If you do a 'ps -x' you will see your btlauncher running in the background. No nohup required. The '2>&1' redirects stderr to stdout which is redirected to torrent.log. This makes it totally silent. Then it's put into the background with '&' and it's backgrounded twice with another '&'. What this does is totally disconnect the process from your terminal. It may look weird, but this is the UNIX idiom that says "not only do I want to run this process asynchronously, I also want the parent process to be the INIT process instead of my terminal". Now, if you actually want to kill the process you can send it a HUP signal 'kill -hup {PID}'.

  • use the screen utility

screen btlaunchmany.py torrent/active/ > torrent.log &

you may exit your terminal session, and when you relogin you can type

screen -r

to reconnect to it.

RH9 users can run this script by may not pick up new torrents as described above.


I use this one as /etc/init.d/bittorrent on debian:

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/bttrack
LAUNCH=/usr/bin/btlaunchmany
MAKEMETA=/usr/bin/btmakemetafile
DFILE=connected.txt
PORT=6969
NAME="bttrack"
DESC="bittorrent tracker"
FILESDIR=/var/torrentfiles
TORRENTSDIR=/var/www/torrents
SERVER=http://www.peix.org:6969

OPTIONS="--dfile ./$DFILE --port $PORT"
test -f $DAEMON || exit 0
cd $FILESDIR
set -e

case "$1" in
  make)
  echo "Making torrents: "
  for file in $FILESDIR/*
  do
    if [[ `basename $file` = "." ]]; then
        continue;
    fi
    if [[ `basename $file` = "$DFILE" ]]; then
        continue;
    fi
    echo $file
    $MAKEMETA $file $SERVER/announce
    cp $FILESDIR/*.torrent $TORRENTSDIR
  done
  echo "."
  ;;
 start)
  echo -n "Starting $DESC: $NAME"
  start-stop-daemon --oknodo -S -b -x $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  stop)
  echo -n "Stopping $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  echo "."
  ;;
  restart|force-reload)
  echo "Restarting $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  start-stop-daemon --oknodo -S -b -x  $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  *)
  echo "Usage: $0 {start|stop|restart|force-reload|make}" >&2
  exit 1
  ;;
esac

exit 0

hope it helps


I, mailto:claw+torrent@kanga.nu, made some slight modifications to the above so as to support multiple source directories for the files that will be torrented:

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/bttrack
LAUNCH=/usr/bin/btlaunchmany
MAKEMETA=/usr/bin/btmakemetafile
DFILE=connected.txt
PORT=6969
NAME="bttrack"
DESC="bittorrent tracker"
TORRENTSDIR=/var/www/downloads/torrents
SERVER=http://research.warnes.net:6969
DEFAULTS_FILE=/etc/default/bittorrent
OPTIONS="--dfile ./$DFILE --port $PORT"

if [[ -s $DEFAULTS_FILE ]]; then
    . $DEFAULTS_FILE
fi
test -f $DAEMON || exit 0
cd $FILESDIR
set -e

case "$1" in
  make)
  echo "Making torrents: "
  for dir in ${TORRENT_DIRS}
  do
    #rm -f ${dir}/*.torrent
    for file in ${dir}/*
    do
      base=`basename $file`
      if [[ "$base" = "." ]]; then
        continue;
      fi
      if [[ "$base" = "$DFILE" ]]; then
        continue;
      fi
      echo $file
      $MAKEMETA $file $SERVER/announce
    done
    mv ${dir}/*.torrent $TORRENTSDIR
  done
  chown www-data.www-data ${TORRENTSDIR}/*
  echo "."
  ;;
 start)
  echo -n "Starting $DESC: $NAME"
  start-stop-daemon --oknodo -S -b -x $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  stop)
  echo -n "Stopping $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  echo "."
  ;;
  restart|force-reload)
  echo "Restarting $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  start-stop-daemon --oknodo -S -b -x  $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  *)
  echo "Usage: $0 {start|stop|restart|force-reload|make}" >&2
  exit 1
  ;;
esac

exit 0

The main added feature is an /etc/default/bittorrent file which should read something like:

#
# List of directories which can contain torrent files to be served
# by the local system.
#
TORRENT_DIRS="/dir1/dir2/dir3 /dir4/dir5/dir6 /dir7/dir8/dir9"

From: Yuriy Krylov (ykrylov -AT - gmail -DOT- com)

This is a debian-style btlaunchmany.bittorrent which pics off "active" torrents from a specified directory moves the file and torrent to "completed" folder when done and emails the owner a notification of completion.

#!/usr/bin/python

# Written by Michael Janssen (jamuraa at base0 dot net)
# originally heavily borrowed code from btlaunchmany.py by Bram Cohen
# and btdownloadcurses.py written by Henry 'Pi' James
# now not so much.
# fmttime and fmtsize stolen from btdownloadcurses.
# see LICENSE.txt for license information

from BitTorrent.download import download
from threading import Thread, Event, Lock
from os import listdir, rename
from os.path import abspath, join, exists, getsize, basename, isfile
from sys import argv, stdout, exit
from time import sleep
import traceback
import sys
import smtplib

print "btlaunchmany.bittorrent is RUNNING"

LOG = "/home/bittorrent/public_html/log.txt"
FILE = open(LOG,"w+")
old_out = sys.stdout
old_err = sys.stderr
sys.stdout = sys.stderr = FILE

COMPLETED = "/home/bittorrent/completed/"

def cleanup():
        FILE.close()
        sys.stdout = old_out
	sys.stderr = old_err


def sendMail(filename,status):
	short_filename = basename(filename)
	print "SENDING NOTIFICATION ABOUT %s %s" % (status, short_filename)
	sys.stdout.flush()
	sys.stderr.flus()
	server = smtplib.SMTP("localhost")
	server.sendmail("bittorrent@yuriy.org","yuriy@localhost", "Subject: bittorent:%s %s has %s download!" % (short_filename,short_filename,status))
	server.quit()

def cleanTorrent(filename):
	short_filename = basename(filename)
	print "REMOVING TORRENT %s" % short_filename
	sys.stdout.flush()
	sys.stderr.flush()
	rename(filename,COMPLETED + short_filename)
	rename(filename+".torrent",COMPLETED + short_filename + ".torrent")

def fmttime(n):
    if n == -1:
        return '(no seeds?)'
    if n == 0:
        return 'complete'
    n = int(n)
    m, s = divmod(n, 60)
    h, m = divmod(m, 60)
    if h > 1000000:
        return 'n/a'
    return '%d:%02d:%02d' % (h, m, s)

def fmtsize(n, baseunit = 0, padded = 1):
    unit = [' B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']
    i = baseunit
    while i + 1 < len(unit) and n >= 999:
        i += 1
        n = float(n) / (1 << 10)
    size = ''
    if padded:
        if n < 10:
            size = '  '
        elif n < 100:
            size = ' '
    if i != 0:
        size += '%.1f %s' % (n, unit[i])
    elif padded:
        size += '%.0f   %s' % (n, unit[i])
    else:
        size += '%.0f %s' % (n, unit[i])
    return size


def dummy(*args, **kwargs):
    pass

threads = {}
ext = '.torrent'
print 'btlaunchmany starting..'
print '...logging to ' + LOG
filecheck = Lock()

def dropdir_mainloop(d, params):
    deadfiles = []
    while True:
        files = listdir(d)
        # new files
        for file in files:
            if file.endswith(ext) and \:
               file not in threads.keys() + deadfiles:
                threads[file] = {'kill': Event(), 'try': 1}
                print 'New torrent: %s' % file
                sys.stdout.flush()
		sys.stderr.flush()
                threads[file]['thread'] = Thread(target = StatusUpdater(join(d, file), params, file).download, name = file)
                threads[file]['thread'].start()
        # files with multiple tries
        for file, threadinfo in threads.items():
            if threadinfo.get('timeout') == 0:
                # Zero seconds left, try and start the thing again.
                threadinfo['try'] += 1
                threadinfo['thread'] = Thread(target = StatusUpdater(join(d, file), params, file).download, name = file)
                threadinfo['thread'].start()
                threadinfo['timeout'] = -1
            elif threadinfo.get('timeout') > 0:
                # Decrement our counter by 1
                threadinfo['timeout'] -= 1
            elif not threadinfo['thread'].isAlive():
                # died without permission
                # if it was checking the file, it isn't anymore.
                if threadinfo.get('checking'):
                    filecheck.release()
                if threadinfo.get('try') == 6:
                    # Died on the sixth try? You're dead.
                    deadfiles.append(file)
                    print '%s died 6 times, added to dead list' % fil
                    sys.stdout.flush()
                    sys.stderr.flush()
		    del threads[file]
                else:
                    del threadinfo['thread']
                    threadinfo['timeout'] = 10
            # dealing with files that disappear
            if file not in files:
                print 'Torrent file disappeared, killing %s' % file
                stdout.flush()
                if threadinfo.get('timeout', -1) == -1:
                    threadinfo['kill'].set()
                    threadinfo['thread'].join()
                # if this thread was filechecking, open it up
                if threadinfo.get('checking'):
                    filecheck.release()
                del threads[file]
        for file in deadfiles:
            # if the file dissapears, remove it from our dead list
            if file not in files:
                deadfiles.remove(file)
        sleep(1)

def display_thread(displaykiller):
    interval = 1.0
    global status
    while True:
        # display file info
        if displaykiller.isSet():
            break
        totalup = 0
        totaldown = 0
        totaluptotal = 0.0
        totaldowntotal = 0.0
        tdis = threads.items()
        tdis.sort()
        for file, threadinfo in tdis:
            uprate = threadinfo.get('uprate', 0)
            downrate = threadinfo.get('downrate', 0)
            uptxt = fmtsize(uprate, padded = 0)
            downtxt = fmtsize(downrate, padded = 0)
            uptotal = threadinfo.get('uptotal', 0.0)
            downtotal = threadinfo.get('downtotal', 0.0)
            uptotaltxt = fmtsize(uptotal, baseunit = 2, padded = 0)
            downtotaltxt = fmtsize(downtotal, baseunit = 2, padded = 0)
            filename = threadinfo.get('savefile', file)
            if threadinfo.get('timeout', 0) > 0:
                trys = threadinfo.get('try', 1)
                timeout = threadinfo.get('timeout')
                print '%s: try %d died, retry in %d' % (basename(filename), trys, timeout)
            else:
                status = threadinfo.get('status','')
                print '%s: Spd: %s/s:%s/s Tot: %s:%s [%s]' % (basename(filename), uptxt, downtxt, uptotaltxt, downtotaltxt, status)
		if status == 'complete' and isfile(filename):
			sendMail(filename,status)
			cleanTorrent(filename)
            totalup += uprate
            totaldown += downrate
            totaluptotal += uptotal
            totaldowntotal += downtotal
        # display totals line
        totaluptxt = fmtsize(totalup, padded = 0)
        totaldowntxt = fmtsize(totaldown, padded = 0)
        totaluptotaltxt = fmtsize(totaluptotal, baseunit = 2, padded = 0)
        totaldowntotaltxt = fmtsize(totaldowntotal, baseunit = 2, padded = 0)
        print 'All: Spd: %s/s:%s/s Tot: %s:%s' % (totaluptxt, totaldowntxt, totaluptotaltxt, totaldowntotaltxt)
        print
        sys.stdout.flush()
        sys.stderr.flush()
	sleep(interval)

class StatusUpdater:
    def __init__(self, file, params, name):
        self.file = file
        self.params = params
        self.name = name
        self.myinfo = threads[name]
        self.done = 0
        self.checking = 0
        self.activity = 'starting'
        self.display()
        self.myinfo['errors'] = []

    def download(self):
        download(self.params + ['--responsefile', self.file], self.choose, self.display, self.finished, self.err, self.myinfo['kill'], 80)
        print 'Torrent %s stopped' % self.file
        sys.stdout.flush()

    def finished(self):
        self.done = 1
        self.myinfo['done'] = 1
        self.activity = 'complete'
        self.display({'fractionDone' : 1, 'downRate' : 0})

    def err(self, msg):
        self.myinfo['errors'].append(msg)
        self.display()

    def failed(self):
        self.activity = 'failed'
        self.display()

    def choose(self, default, size, saveas, dir):
        self.myinfo['downfile'] = default
        self.myinfo['filesize'] = fmtsize(size)
        if saveas == '':
            saveas = default
        # it asks me where I want to save it before checking the file..
        if exists(self.file[:-len(ext)]) and getsize(self.file[:-len(ext)]) > 0:
            # file will get checked
            while not filecheck.acquire(0) and not self.myinfo['kill'].isSet():
                self.myinfo['status'] = 'disk wait'
                sleep(0.1)
            if not self.myinfo['kill'].isSet():
                self.checking = 1
                self.myinfo['checking'] = 1
        self.myinfo['savefile'] = self.file[:-len(ext)]
        return self.file[:-len(ext)]

    def display(self, dict = {}):
        fractionDone = dict.get('fractionDone')
        timeEst = dict.get('timeEst')
        activity = dict.get('activity')
        if activity is not None and not self.done:
            if activity == 'checking existing file':
                self.activity = 'disk check'
            elif activity == 'connecting to peers':
                self.activity = 'connecting'
            else:
                self.activity = activity
        elif timeEst is not None:
            self.activity = fmttime(timeEst)
        if fractionDone is not None:
            self.myinfo['status'] = '%s %.0f%%' % (self.activity, fractionDone * 100)
        else:
            self.myinfo['status'] = self.activity
        if self.activity != 'checking existing file' and self.checking:
            # we finished checking our files.
            filecheck.release()
            self.checking = 0
            self.myinfo['checking'] = 0
        if 'upRate' in dict:
            self.myinfo['uprate'] = dict['upRate']
        if 'downRate' in dict:
            self.myinfo['downrate'] = dict['downRate']
        if 'upTotal' in dict:
            self.myinfo['uptotal'] = dict['upTotal']
        if 'downTotal' in dict:
            self.myinfo['downtotal'] = dict['downTotal']

if __name__ == '__main__':
    if len(argv) < 2:
        print """Usage: btlaunchmany.py <directory> <global options>
  <directory> - directory to look for .torrent files (non-recursive)
  <global options> - options to be applied to all torrents (see btdownloadheadless.py)
"""
        exit(-1)
    try:
        displaykiller = Event()
        displaythread = Thread(target = display_thread, name = 'display', args = [displaykiller])
        displaythread.start()
        dropdir_mainloop(argv[1], argv[2:])
    except KeyboardInterrupt:
        print '^C caught! Killing torrents..'
        for file, threadinfo in threads.items():
            status = 'Killing torrent %s' % file
            threadinfo['kill'].set()
            threadinfo['thread'].join()
            del threads[file]
        displaykiller.set()
        displaythread.join()
	cleanup()
    except:
        traceback.print_exc()

This is called by /etc/init.d/bittorrent which looks like

#!/bin/bash

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/bttrack
LAUNCH=/usr/bin/btlaunchmany
MAKEMETA=/usr/bin/btmakemetafile
DFILE=connected.txt
LOG=${your_log}
PORT=${your_port}
NAME="bttrack"
DESC="bittorrent tracker"
FILESDIR=/home/bittorrent/active
TORRENTSDIR=/var/www/torrents
SERVER=http://${your_sever}:${your_port}

OPTIONS="--dfile ./$DFILE --port $PORT"
test -f $DAEMON || exit 0
cd $FILESDIR
set -e

case "$1" in
  make)
  echo "Making torrents: "
  for file in $FILESDIR/*
  do
    if [[ `basename $file` = "." ]]; then
        continue;
    fi
    if [[ `basename $file` = "$DFILE" ]]; then
        continue;
    fi
    echo $file
    $MAKEMETA $file $SERVER/announce
    cp $FILESDIR/*.torrent $TORRENTSDIR
  done
  echo "."
  ;;
 start)
  echo -n "Starting $DESC: $NAME"
  start-stop-daemon --oknodo -S -b -x $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  stop)
  echo -n "Stopping $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  echo "."
  ;;
  restart|force-reload)
  echo "Restarting $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  start-stop-daemon --oknodo -S -b -x  $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  *)
  echo "Usage: $0 {start|stop|restart|force-reload|make}" >&2
  exit 1
  ;;
esac

exit 0

The log file gets rotated by cron.hourly. So in /etc/init.d/logrotate.d, create logrotate which rotates the files every hour as long as the log reached 1M:

{$path_to}/log.txt {
        missingok
        rotate 3
        size 1M
	create 775 bittorrent bittorrent
        prerotate
                /etc/init.d/bittorrent stop
	endscript
        postrotate
                /etc/init.d/bittorrent start
        endscript
	nocompress
	notifempty
}

Finally, tell cron to activate your logrotate script by placing within /etc/cron.hourly/logrotate:

#!/bin/sh

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

Hope this helps. -Yuriy


Part of [[]]


Last edit: Fri, 17 Mar 2006 19:24:26 -0800
(%C0%F1%ED%E7%E9%E0%D6%E3%E6%F4%DF%F8%FC)
Revisions: 20