New helper daemon : transmission-daemon-watcher !

[bernstein]

hi everyone

i have written a little daemon for all those running transmission-daemon on a linux (especially ubuntu 8.04) box.

as you all know transmission-daemon has yet to become as powerful as transmission-cli and the GUIs... so i was missing some features so badly i wrote a python daemon that makes using transmission-daemon a snap!

Features :
* Watch a given directory for torrent files and add them to transmission-daemon.
* Optionally remove added torrent files.
* Watch transmission-daemon for completed torrents and remove them at a given seeding ratio.
* Optionally move completed (& from transmission-daemon removed) datafiles.
* Optionally send an email notification upon completion of download
* Watch specified feeds for new torrents (and auto-add new torrents)

link : http://tdw.blunet.cc

enjoy
claude

[Jordan]

Looks like fun! I'll have to give it a spin.

If I'm reading the "minimum requirements" list correctly, this requires Linux's inotify services, so it won't work on OS X. Is that correct?

[bernstein]

sadly YES... i know that OS X 10.5 has introduced a similar feature (fsenvents?!) but unless someone points me to a python module providing the necessary bindings it's a no-go for OS X...

[edit] ok a short google reveals said module : PyObjC if there is demand for this to work on OS X i'll have a look at it and if it's simple enough i'll integrate it.

[mezz]

http://mark.heily.com/pnotify/

Kernel-based filesystem event notification is a relatively new phenonemon, and several different approaches have been taken by various operating systems. Linux uses inotify, BSD uses kqueue, Apple has kqueue and it's own Spotlight system calls, Solaris has FEM, Windows has the NTFS Change Journal and the ReadDirectoryChangesW function, and so forth.

The goal of pnotify is to provide a simple, cross-platform API for monitoring filesystem events. The library will conceal all of the messy implementation-specific details from the user and help them to write more portable programs.

or

http://ox.eicat.ca/~dgilbert/files/PyKQueue.html

PyKQueue is a Python module that provides an object-oriented interface to the FreeBSD syscalls kqueue(2) and kevent(2).

But I don't know if MacOS X and FreeBSD have same exactly kqueue unless MacOS X change too much. The pnotify looks good to fit in your project.

[ximon]

I was excited about this until I realised that my Synology box doesn't have inotify.

[bernstein]

well due the fact that obviously people aren't interested in this tool (and the features it adds to transmission) i have indefinitely posponed (until at least a few people ask for it) any support/features for anything else that i need. This means it runs only on Linux with Kernels that support inotify (2.6.13+). It uses pyinotify to access inotify which supports ANY kernel version with inotify support - however i have not tested this.

Alas, according to this info http://www.synology.com/enu/forum/viewt ... f=3&t=9271(a quick google always helps) the synology boxes should have a 2.6.15 kernel (that should include inotify). Maybe synology has stripped inotify from their kernels?
And they are talking about releasing a new firmware image with a recent kernel version in september 08.

additional googling reveals that at least for synology boxes DS101g+, DS107, DS107+, DS107e, DS207, DS207+, CS407, CS407e, RS407 the source package contains support for inotify (http://gpl.nas-central.org/SYNOLOGY/ & http://synology.nas-central.org/) so in theory it should be possible to create your own firmware image that supports inotify. however do not do this unless you know what you are doing. danger of a bricked device...

[ximon]

I'm interested! Very interested! transmission-dw could add a lot of the functionality that I feel is missing from the running transmission in daemon mode, especially stopping/moving completed downloads. I already use pytvshows for automatically downloading from feeds.

I have been looking into getting it running on my Synology (please bear with me, I'm a relative novice). It seems I was mistaken about inotify, it is indeed built-in to my box.

I manually installed transmission-dw and it's associated tools and I was making progress but I'm pretty sure I've hit a brick wall. It seems that ARM-based boxes are missing libffi ffi_closure support which in turn is preventing Python from being able to use the _ctypes module. Without this pyinotify won't run, and without pyinotify transmission-dw won't work.

D'oh! I have but a very basic knowledge of programming, is this something that I can get around or am I done for?

[bernstein]

well i'm not that experienced in programming... otherwise i should have seen this

a short poking around reveals that indeed ctypes won't work. the most recent info on the matter i found is here : http://mail.python.org/pipermail/python ... 98073.html. Reading other sources show that libffi 3.0 was released in Feb.08 and should support the missing features on ARM. however i think that python uses an older custom fork of libffi in the ctypes module which has not complete support for ARM.
* so it may be that the upcoming python 3.0 will work...
* it might also be possible to stip together a firmware image with a patched ctypes module using code from the latest libffi release....
but then that would boil down to a new firmware... OR rewriting transmission-dw in C/C++ but as i too have next to none expericence coding C/C++ sadly it's a no go (otherwise i would have made patches for transmission itself)

probably the easiest solution :
* it might be possible to use an older ~0.7 version of the pyinotify module as this doesn't depend on ctypes! however this would require you to compile the module for ARM - if there isn't already one on the net. Chances are this would just work, but it might bring up even more problems.

oh, my i just realize : if you don't need the pyinotify support (auto adding new torrents by watching a directory) there is NO need for ctypes!!! however since this brought the necessity of ctypes i have used it to conveniently refuse running multiple instances of transmission-dw... if you know enough python you can stip those requirements away. else i offer to make the changes - however i have zero time right now, i could do it till end of september.

Edit : using without ctypes would also break the current feed support (but that could be fixed - if needed)

[ximon]

Thanks for the info. I don't think you have grasped just how basic my programming knowledge is - I'll have a go at modifying the python script, but I am an absolute beginner as far as python is concerned so we'll see what happens...

I take it that stopping/moving completed torrents is not dependent on inotify, but rather is polling transmission-daemon for status?

I already have a shell script in crontab that watches a folder and adds torrents to the daemon as well as pytvshows for feeds, so I don't really need the functionality that removing ctypes would break (although it would have been nice to have one script to rule them all!).

I'll keep you posted on how I get on, but I have a sneaking suspicion I'm already in way over my head. Still, this is how we learn things I suppose...

[bernstein]

I take it that stopping/moving completed torrents is not dependent on inotify, but rather is polling transmission-daemon for status?

not quite: yes to stop/move stuff polling transmission-daemon is necessary. but to keep things simple (and to get results fast) it just uses transmission-remote. i plan on doing it through transmissions own protocol but by the time i get to that transmission itself may have the features itself. pyinotify is only used to detect files added to a given directory. for simplicity the feed stuff depends on this (it basically gets a feed, sorts out the new entries and downloads the respective torrent files to the watched directory)

one more thing: commenting lines 102-119, 252-257 and removing the pyinotify item in line 35 should remove the pyinotify dependency. after that there should only the set_process_title() funtion in proccontrol.py use ctypes. the easiest way would be to comment lines 13-15 and insert one after line 12 saying 'pass'. note that it has to preceded by a <tab>. this will however remove the ability to detect if is already running (and replacing that instance instead of creating a new one)...

[ximon]

Thanks for that, I'll give it a try over the weekend. I thought that transmission-remote was incompatible with Clutch though, but I may have imagined that.

[Jordan]

I'm not sure what `incompatible' means here. Clutch and transmission-remote serve exactly the same purpose -- they're both user interfaces to manipulate a running transmission session (either the daemon, or gtk client, or mac client). Clutch and transmission-remote are interchangeable.

[ximon]

OK, I thought Clutch played by its own rules since it ignored the folders specified in the .conf file and wouldn't play nice with transmission-watchdog. But as I said, my understanding of these things is limited, I just like to tinker around with them.

[cerbero]

Thanks so much for this! This will probably finally let me replace rtorrent+wtorrent with Transmission.

[ajf88]

Good work! I used to have a similar way of managing things (over NFS or samba) with btlaunchmanycurses and it worked fairly well, but I much prefer transmission over bittornado. I'll carry on using my windows RPC client (which I've made a bunch of updates to btw), but this is a very tidy and importantly platform neutral way of doing things when you use some network filesystem. If I was using UNIX on my laptop (wlan interface not supported) I'd probably use this, as my GTK/ncurses skills aren't great.

I imagine using directory polling wouldn't be a huge performance hit for the people without these fs event features...

I'm very interested in writing a ncurses client, but I have only had a little time and progress.