BASH script to run bittorrent as a daemon

From Theory.org Wiki
Jump to: navigation, search

By using the btlaunchmany.py executable you can do this:

I created a torrent directory which only contains two other directories: active and standby I then download all .torrent file into the standby directory and when I want to start fetching the file I simply move the file over to the active directory.

I start the BitTorrent executable as a background task using this script:

#!/bin/sh
cd
nohup btlaunchmany.py torrent/active/ > torrent.log &
tail -f torrent.log

It will start the BitTorrent and scans the active directory for all *.torrent references and will then download the files in the same directory. Fortunately the btlaunchmany.py is scanning the active directory frequently, so by simply moving a *.torrent file into that directory it will be recognized and a new download thread will be created.

To avoid this process exiting when you log out, you can either

  • use nohup

nohup btlaunchmany.py torrent/active/ > torrent.log &

  • double background it
    (./btlaunchmany.py torrent/active/ > torrent.log 2>&1 &) &

This will display a message something like this:

    [1] + Done                 ( ./btlaunchmany.py torrent/active/ > torrent.l

That's OK. If you do a 'ps -x' you will see your btlauncher running in the background. No nohup required. The '2>&1' redirects stderr to stdout which is redirected to torrent.log. This makes it totally silent. Then it's put into the background with '&' and it's backgrounded twice with another '&'. What this does is totally disconnect the process from your terminal. It may look weird, but this is the UNIX idiom that says "not only do I want to run this process asynchronously, I also want the parent process to be the INIT process instead of my terminal". Now, if you actually want to kill the process you can send it a HUP signal 'kill -hup {PID}'.

  • use the screen utility

screen btlaunchmany.py torrent/active/ > torrent.log &

you may exit your terminal session, and when you relogin you can type

screen -r

to reconnect to it.

RH9 users can run this script by may not pick up new torrents as described above.


I use this one as /etc/init.d/bittorrent on debian:

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/bttrack
LAUNCH=/usr/bin/btlaunchmany
MAKEMETA=/usr/bin/btmakemetafile
DFILE=connected.txt
PORT=6969
NAME="bttrack"
DESC="bittorrent tracker"
FILESDIR=/var/torrentfiles
TORRENTSDIR=/var/www/torrents
SERVER=http://www.peix.org:6969

OPTIONS="--dfile ./$DFILE --port $PORT"
test -f $DAEMON || exit 0
cd $FILESDIR
set -e

case "$1" in
  make)
  echo "Making torrents: "
  for file in $FILESDIR/*
  do
    if [[ `basename $file` = "." ]]; then
        continue;
    fi
    if [[ `basename $file` = "$DFILE" ]]; then
        continue;
    fi
    echo $file
    $MAKEMETA $file $SERVER/announce
    cp $FILESDIR/*.torrent $TORRENTSDIR
  done
  echo "."
  ;;
 start)
  echo -n "Starting $DESC: $NAME"
  start-stop-daemon --oknodo -S -b -x $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  stop)
  echo -n "Stopping $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  echo "."
  ;;
  restart|force-reload)
  echo "Restarting $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  start-stop-daemon --oknodo -S -b -x  $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  *)
  echo "Usage: $0 {start|stop|restart|force-reload|make}" >&2
  exit 1
  ;;
esac

exit 0

hope it helps


I, mailto:claw+torrent@kanga.nu, made some slight modifications to the above so as to support multiple source directories for the files that will be torrented:

#! /bin/sh

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/bttrack
LAUNCH=/usr/bin/btlaunchmany
MAKEMETA=/usr/bin/btmakemetafile
DFILE=connected.txt
PORT=6969
NAME="bttrack"
DESC="bittorrent tracker"
TORRENTSDIR=/var/www/downloads/torrents
SERVER=http://research.warnes.net:6969
DEFAULTS_FILE=/etc/default/bittorrent
OPTIONS="--dfile ./$DFILE --port $PORT"

if [[ -s $DEFAULTS_FILE ]]; then
    . $DEFAULTS_FILE
fi
test -f $DAEMON || exit 0
cd $FILESDIR
set -e

case "$1" in
  make)
  echo "Making torrents: "
  for dir in ${TORRENT_DIRS}
  do
    #rm -f ${dir}/*.torrent
    for file in ${dir}/*
    do
      base=`basename $file`
      if [[ "$base" = "." ]]; then
        continue;
      fi
      if [[ "$base" = "$DFILE" ]]; then
        continue;
      fi
      echo $file
      $MAKEMETA $file $SERVER/announce
    done
    mv ${dir}/*.torrent $TORRENTSDIR
  done
  chown www-data.www-data ${TORRENTSDIR}/*
  echo "."
  ;;
 start)
  echo -n "Starting $DESC: $NAME"
  start-stop-daemon --oknodo -S -b -x $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  stop)
  echo -n "Stopping $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  echo "."
  ;;
  restart|force-reload)
  echo "Restarting $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  start-stop-daemon --oknodo -S -b -x  $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  *)
  echo "Usage: $0 {start|stop|restart|force-reload|make}" >&2
  exit 1
  ;;
esac

exit 0

The main added feature is an /etc/default/bittorrent file which should read something like:

#
# List of directories which can contain torrent files to be served
# by the local system.
#
TORRENT_DIRS="/dir1/dir2/dir3 /dir4/dir5/dir6 /dir7/dir8/dir9"

From: Yuriy Krylov (ykrylov -AT - gmail -DOT- com)

This is a debian-style btlaunchmany.bittorrent which pics off "active" torrents from a specified directory moves the file and torrent to "completed" folder when done and emails the owner a notification of completion.

#!/usr/bin/python

# Written by Michael Janssen (jamuraa at base0 dot net)
# originally heavily borrowed code from btlaunchmany.py by Bram Cohen
# and btdownloadcurses.py written by Henry 'Pi' James
# now not so much.
# fmttime and fmtsize stolen from btdownloadcurses.
# see LICENSE.txt for license information

from BitTorrent.download import download
from threading import Thread, Event, Lock
from os import listdir, rename
from os.path import abspath, join, exists, getsize, basename, isfile
from sys import argv, stdout, exit
from time import sleep
import traceback
import sys
import smtplib

print "btlaunchmany.bittorrent is RUNNING"

LOG = "/home/bittorrent/public_html/log.txt"
FILE = open(LOG,"w+")
old_out = sys.stdout
old_err = sys.stderr
sys.stdout = sys.stderr = FILE

COMPLETED = "/home/bittorrent/completed/"

def cleanup():
        FILE.close()
        sys.stdout = old_out
	sys.stderr = old_err


def sendMail(filename,status):
	short_filename = basename(filename)
	print "SENDING NOTIFICATION ABOUT %s %s" % (status, short_filename)
	sys.stdout.flush()
	sys.stderr.flus()
	server = smtplib.SMTP("localhost")
	server.sendmail("bittorrent@yuriy.org","yuriy@localhost", "Subject: bittorent:%s %s has %s download!" % (short_filename,short_filename,status))
	server.quit()

def cleanTorrent(filename):
	short_filename = basename(filename)
	print "REMOVING TORRENT %s" % short_filename
	sys.stdout.flush()
	sys.stderr.flush()
	rename(filename,COMPLETED + short_filename)
	rename(filename+".torrent",COMPLETED + short_filename + ".torrent")

def fmttime(n):
    if n == -1:
        return '(no seeds?)'
    if n == 0:
        return 'complete'
    n = int(n)
    m, s = divmod(n, 60)
    h, m = divmod(m, 60)
    if h > 1000000:
        return 'n/a'
    return '%d:%02d:%02d' % (h, m, s)

def fmtsize(n, baseunit = 0, padded = 1):
    unit = [' B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']
    i = baseunit
    while i + 1 < len(unit) and n >= 999:
        i += 1
        n = float(n) / (1 << 10)
    size = ''
    if padded:
        if n < 10:
            size = '  '
        elif n < 100:
            size = ' '
    if i != 0:
        size += '%.1f %s' % (n, unit[i])
    elif padded:
        size += '%.0f   %s' % (n, unit[i])
    else:
        size += '%.0f %s' % (n, unit[i])
    return size


def dummy(*args, **kwargs):
    pass

threads = {}
ext = '.torrent'
print 'btlaunchmany starting..'
print '...logging to ' + LOG
filecheck = Lock()

def dropdir_mainloop(d, params):
    deadfiles = []
    while True:
        files = listdir(d)
        # new files
        for file in files:
            if file.endswith(ext) and \:
               file not in threads.keys() + deadfiles:
                threads[file] = {'kill': Event(), 'try': 1}
                print 'New torrent: %s' % file
                sys.stdout.flush()
		sys.stderr.flush()
                threads[file]['thread'] = Thread(target = StatusUpdater(join(d, file), params, file).download, name = file)
                threads[file]['thread'].start()
        # files with multiple tries
        for file, threadinfo in threads.items():
            if threadinfo.get('timeout') == 0:
                # Zero seconds left, try and start the thing again.
                threadinfo['try'] += 1
                threadinfo['thread'] = Thread(target = StatusUpdater(join(d, file), params, file).download, name = file)
                threadinfo['thread'].start()
                threadinfo['timeout'] = -1
            elif threadinfo.get('timeout') > 0:
                # Decrement our counter by 1
                threadinfo['timeout'] -= 1
            elif not threadinfo['thread'].isAlive():
                # died without permission
                # if it was checking the file, it isn't anymore.
                if threadinfo.get('checking'):
                    filecheck.release()
                if threadinfo.get('try') == 6:
                    # Died on the sixth try? You're dead.
                    deadfiles.append(file)
                    print '%s died 6 times, added to dead list' % fil
                    sys.stdout.flush()
                    sys.stderr.flush()
		    del threads[file]
                else:
                    del threadinfo['thread']
                    threadinfo['timeout'] = 10
            # dealing with files that disappear
            if file not in files:
                print 'Torrent file disappeared, killing %s' % file
                stdout.flush()
                if threadinfo.get('timeout', -1) == -1:
                    threadinfo['kill'].set()
                    threadinfo['thread'].join()
                # if this thread was filechecking, open it up
                if threadinfo.get('checking'):
                    filecheck.release()
                del threads[file]
        for file in deadfiles:
            # if the file dissapears, remove it from our dead list
            if file not in files:
                deadfiles.remove(file)
        sleep(1)

def display_thread(displaykiller):
    interval = 1.0
    global status
    while True:
        # display file info
        if displaykiller.isSet():
            break
        totalup = 0
        totaldown = 0
        totaluptotal = 0.0
        totaldowntotal = 0.0
        tdis = threads.items()
        tdis.sort()
        for file, threadinfo in tdis:
            uprate = threadinfo.get('uprate', 0)
            downrate = threadinfo.get('downrate', 0)
            uptxt = fmtsize(uprate, padded = 0)
            downtxt = fmtsize(downrate, padded = 0)
            uptotal = threadinfo.get('uptotal', 0.0)
            downtotal = threadinfo.get('downtotal', 0.0)
            uptotaltxt = fmtsize(uptotal, baseunit = 2, padded = 0)
            downtotaltxt = fmtsize(downtotal, baseunit = 2, padded = 0)
            filename = threadinfo.get('savefile', file)
            if threadinfo.get('timeout', 0) > 0:
                trys = threadinfo.get('try', 1)
                timeout = threadinfo.get('timeout')
                print '%s: try %d died, retry in %d' % (basename(filename), trys, timeout)
            else:
                status = threadinfo.get('status','')
                print '%s: Spd: %s/s:%s/s Tot: %s:%s [%s]' % (basename(filename), uptxt, downtxt, uptotaltxt, downtotaltxt, status)
		if status == 'complete' and isfile(filename):
			sendMail(filename,status)
			cleanTorrent(filename)
            totalup += uprate
            totaldown += downrate
            totaluptotal += uptotal
            totaldowntotal += downtotal
        # display totals line
        totaluptxt = fmtsize(totalup, padded = 0)
        totaldowntxt = fmtsize(totaldown, padded = 0)
        totaluptotaltxt = fmtsize(totaluptotal, baseunit = 2, padded = 0)
        totaldowntotaltxt = fmtsize(totaldowntotal, baseunit = 2, padded = 0)
        print 'All: Spd: %s/s:%s/s Tot: %s:%s' % (totaluptxt, totaldowntxt, totaluptotaltxt, totaldowntotaltxt)
        print
        sys.stdout.flush()
        sys.stderr.flush()
	sleep(interval)

class StatusUpdater:
    def __init__(self, file, params, name):
        self.file = file
        self.params = params
        self.name = name
        self.myinfo = threads[name]
        self.done = 0
        self.checking = 0
        self.activity = 'starting'
        self.display()
        self.myinfo['errors'] = []

    def download(self):
        download(self.params + ['--responsefile', self.file], self.choose, self.display, self.finished, self.err, self.myinfo['kill'], 80)
        print 'Torrent %s stopped' % self.file
        sys.stdout.flush()

    def finished(self):
        self.done = 1
        self.myinfo['done'] = 1
        self.activity = 'complete'
        self.display({'fractionDone' : 1, 'downRate' : 0})

    def err(self, msg):
        self.myinfo['errors'].append(msg)
        self.display()

    def failed(self):
        self.activity = 'failed'
        self.display()

    def choose(self, default, size, saveas, dir):
        self.myinfo['downfile'] = default
        self.myinfo['filesize'] = fmtsize(size)
        if saveas == '':
            saveas = default
        # it asks me where I want to save it before checking the file..
        if exists(self.file[:-len(ext)]) and getsize(self.file[:-len(ext)]) > 0:
            # file will get checked
            while not filecheck.acquire(0) and not self.myinfo['kill'].isSet():
                self.myinfo['status'] = 'disk wait'
                sleep(0.1)
            if not self.myinfo['kill'].isSet():
                self.checking = 1
                self.myinfo['checking'] = 1
        self.myinfo['savefile'] = self.file[:-len(ext)]
        return self.file[:-len(ext)]

    def display(self, dict = {}):
        fractionDone = dict.get('fractionDone')
        timeEst = dict.get('timeEst')
        activity = dict.get('activity')
        if activity is not None and not self.done:
            if activity == 'checking existing file':
                self.activity = 'disk check'
            elif activity == 'connecting to peers':
                self.activity = 'connecting'
            else:
                self.activity = activity
        elif timeEst is not None:
            self.activity = fmttime(timeEst)
        if fractionDone is not None:
            self.myinfo['status'] = '%s %.0f%%' % (self.activity, fractionDone * 100)
        else:
            self.myinfo['status'] = self.activity
        if self.activity != 'checking existing file' and self.checking:
            # we finished checking our files.
            filecheck.release()
            self.checking = 0
            self.myinfo['checking'] = 0
        if 'upRate' in dict:
            self.myinfo['uprate'] = dict['upRate']
        if 'downRate' in dict:
            self.myinfo['downrate'] = dict['downRate']
        if 'upTotal' in dict:
            self.myinfo['uptotal'] = dict['upTotal']
        if 'downTotal' in dict:
            self.myinfo['downtotal'] = dict['downTotal']

if __name__ == '__main__':
    if len(argv) < 2:
        print """Usage: btlaunchmany.py <directory> <global options>
  <directory> - directory to look for .torrent files (non-recursive)
  <global options> - options to be applied to all torrents (see btdownloadheadless.py)
"""
        exit(-1)
    try:
        displaykiller = Event()
        displaythread = Thread(target = display_thread, name = 'display', args = [displaykiller])
        displaythread.start()
        dropdir_mainloop(argv[1], argv[2:])
    except KeyboardInterrupt:
        print '^C caught! Killing torrents..'
        for file, threadinfo in threads.items():
            status = 'Killing torrent %s' % file
            threadinfo['kill'].set()
            threadinfo['thread'].join()
            del threads[file]
        displaykiller.set()
        displaythread.join()
	cleanup()
    except:
        traceback.print_exc()

This is called by /etc/init.d/bittorrent which looks like

#!/bin/bash

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/bttrack
LAUNCH=/usr/bin/btlaunchmany
MAKEMETA=/usr/bin/btmakemetafile
DFILE=connected.txt
LOG=${your_log}
PORT=${your_port}
NAME="bttrack"
DESC="bittorrent tracker"
FILESDIR=/home/bittorrent/active
TORRENTSDIR=/var/www/torrents
SERVER=http://${your_sever}:${your_port}

OPTIONS="--dfile ./$DFILE --port $PORT"
test -f $DAEMON || exit 0
cd $FILESDIR
set -e

case "$1" in
  make)
  echo "Making torrents: "
  for file in $FILESDIR/*
  do
    if [[ `basename $file` = "." ]]; then
        continue;
    fi
    if [[ `basename $file` = "$DFILE" ]]; then
        continue;
    fi
    echo $file
    $MAKEMETA $file $SERVER/announce
    cp $FILESDIR/*.torrent $TORRENTSDIR
  done
  echo "."
  ;;
 start)
  echo -n "Starting $DESC: $NAME"
  start-stop-daemon --oknodo -S -b -x $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  stop)
  echo -n "Stopping $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  echo "."
  ;;
  restart|force-reload)
  echo "Restarting $DESC: $NAME"
  start-stop-daemon --oknodo -K -q -R 30 -n $NAME
  start-stop-daemon --oknodo -K -q -R 30 -n `basename $LAUNCH`
  start-stop-daemon --oknodo -S -b -x  $DAEMON -- $OPTIONS
  start-stop-daemon --oknodo -S -b -x $LAUNCH -- $FILESDIR
  echo "."
  ;;
  *)
  echo "Usage: $0 {start|stop|restart|force-reload|make}" >&2
  exit 1
  ;;
esac

exit 0

The log file gets rotated by cron.hourly. So in /etc/init.d/logrotate.d, create logrotate which rotates the files every hour as long as the log reached 1M:

{$path_to}/log.txt {
        missingok
        rotate 3
        size 1M
	create 775 bittorrent bittorrent
        prerotate
                /etc/init.d/bittorrent stop
	endscript
        postrotate
                /etc/init.d/bittorrent start
        endscript
	nocompress
	notifempty
}

Finally, tell cron to activate your logrotate script by placing within /etc/cron.hourly/logrotate:

#!/bin/sh

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

Hope this helps. -Yuriy


Part of [[]]


Last edit: Fri, 17 Mar 2006 19:24:26 -0800
(%C0%F1%ED%E7%E9%E0%D6%E3%E6%F4%DF%F8%FC)
Revisions: 20