f_nocache option

[otterley]

Would it be possible to add an option (perhaps in the UI) to have Transmission perform an fcntl(fd, F_NOCACHE) on files that that are being shared/retrieved?

Currently Transmission (like all BT programs) thrashes my buffer cache, because it's randomly seeking over many gigabytes of files it is sharing/storing that are unlikely ever to be read again. Caching works well if the working set of files being shared/downloaded is less than the RAM in the machine, but anything over that and my system slows to a crawl due to excessive paging. (The OS will page out background programs to make room for the buffer cache -- to demonstrate the effect, try sharing 16GB of heavily-trafficked files on a 4GB system, go away for a few hours, letting the screen saver kick in, and then log back in.)

[Stefan_Fisk]

I hacked together a build of v1.42 that sets F_NOCACHE or all files by adding the code below to TrOpenFile(). I could not notice any performance hit in Transmission, but leaving Transmission open no longer results in heavy swapping.

I can't say that I understand the full impact of this to well, but as far as my opinion goes this should be the default behaviour for libtransmission under Mac OS X.

If anyone wants a copy of the build just contact me..

Code: Select all

#if defined(SYS_DARWIN)
	
    if( fcntl(file->fd, F_NOCACHE, 1) == -1 )
    {
        const int err = errno;
        tr_err( _( "Couldn't set F_NOCACHE for \"%1$s\": %2$s" ), filename,
               tr_strerror( err ) );
        tr_free( filename );
        return err;
    }

#endif

[Jordan]

I've added this to r7798, though I think it's better for F_NOCACHE to fail silently (as r7798 does) than to refuse to open the file (as in the above patch). Also added in r7798 for Mac users is an fcntl hint to disable the read-ahead buffer, since local data is typically accessed in a near-random method.

I'd be interested in getting feedback from Mac users on what effect (if any) this has in the nightlies...

[otterley]

I examined your changeset. Thanks!

However, I think we want POSIX_FADV_DONTNEED (i.e., don't cache) instead of POSIX_FADV_RANDOM (i.e., don't readahead) on Linux. Alternatively, consider opening the fd with O_DIRECT semantics.

[Jordan]

I examined your changeset. Thanks!

However, I think we want POSIX_FADV_DONTNEED (i.e., don't cache) instead of POSIX_FADV_RANDOM (i.e., don't readahead) on Linux. Alternatively, consider opening the fd with O_DIRECT semantics.

There are two separate issues here -- disabling the readahead, and disabling the cache.

Disabling the readahead is handled in that commit with POSIX_FADV_RANDOM.

Disabling the cache seems to be ugly to do in Linux. POSIX_FADV_DONTNEED doesn't disable the cache, but rather purges pages from the cache, so it needs to be called periodically. The O_DIRECT alternative has portability and alignment complications that IMO outweigh the benefits.

Possibly we could call POSIX_FADV_DONTNEED in tr_fdFileReturn()...

[otterley]

Disabling the cache seems to be ugly to do in Linux. POSIX_FADV_DONTNEED doesn't disable the cache, but rather purges pages from the cache, so it needs to be called periodically. The O_DIRECT alternative has portability and alignment complications that IMO outweigh the benefits.

At least under Linux 2.6, writes to files opened with O_DIRECT need only be aligned on 512-byte boundaries. Is this too difficult to adhere to? (Probably best not to bother with the Linux 2.4 constraints.)

[Stefan_Fisk]

May I ask why you think that F_RDAHEAD should be used? With piece sizes from a couple of 100 kB to several MB, is it really wise to completely disable read-ahead?