Disk Cache decrease the read and write frequency of your HD
Disk Cache decrease the read and write frequency of your HD
the disk cache feature which greatly extends the life of hard disk.
Re: Disk Cache decrease the read and write frequency of your HD
I think transmissions already has a disk cache.
At least when I was downloading a file at close to 2 MB/s it didn't write to disk continuously. It wrote to disks at regular intervals, and I noticed it by the download speed dropping. So maybe it's a bit too aggressive since the disk becomes overloaded and Transmission has to slow down for a second?
At least when I was downloading a file at close to 2 MB/s it didn't write to disk continuously. It wrote to disks at regular intervals, and I noticed it by the download speed dropping. So maybe it's a bit too aggressive since the disk becomes overloaded and Transmission has to slow down for a second?
-
exiztone
Re: Disk Cache decrease the read and write frequency of your
-
Last edited by exiztone on Wed Jul 24, 2013 5:57 pm, edited 1 time in total.
-
NitricJerkSud
- Posts: 6
- Joined: Tue Jan 20, 2009 6:54 am
Re: Disk Cache decrease the read and write frequency of your HD
I just popped in a new 320GB 7200RPM drive into my MBP this morning -- the faster drive DOES make a difference. Also, consider the fact that you're going to need as much RAM as your torrent file occupies on disk in order to eliminate READS from disk. (Because you don't want to download a piece, and NOT write it to disk. If you crash, you have to download it again.)pepperpupper wrote:I have a powerbook with a 4200rpm drive, and at about 1mb/second it is almost impossible to use the computer for anything else but Transmission. Utorrent and LH-ABC does not seem to write to the disk as often.
If you're downloading those 125+GB Nine Inch Nails torrents, a RAM cache isn't going to save you...
Re: Disk Cache decrease the read and write frequency of your HD
Hmm I think you have somewhat misuderstood the write/read cache idea. The idea is NOT to cache everything into memory. The idea is to cache bunch of data into memory and then try to read/write it into hardisk in a one go if possible. 7200RPM hard disks does certainly help a little, but even the slowest of slowest modern hard disk can easily keep up with 10-20MB/s read/write speeds . Unfortunately the bigger problem is the harddisk seeking. Transmission is downloading/reading a lot of pieces from different parts of files which causes the hard disk to seek up & down all the time causing the read/write throughput to collapse. And this is the part where the cache steps in.NitricJerkSud wrote:I just popped in a new 320GB 7200RPM drive into my MBP this morning -- the faster drive DOES make a difference. Also, consider the fact that you're going to need as much RAM as your torrent file occupies on disk in order to eliminate READS from disk. (Because you don't want to download a piece, and NOT write it to disk. If you crash, you have to download it again.)pepperpupper wrote:I have a powerbook with a 4200rpm drive, and at about 1mb/second it is almost impossible to use the computer for anything else but Transmission. Utorrent and LH-ABC does not seem to write to the disk as often.
If you're downloading those 125+GB Nine Inch Nails torrents, a RAM cache isn't going to save you...
For example I have a 110M Internet connection and Transmission is the limiting factor in download speeds. I can only get around 3-4MB/s download speeds into my external USB harddisk with Transmission because it has either poor or too small write/read-caching.
Azureus does a lot of better after little bit of tweaking. I set the cache size to 64MB and enabled option to asking Azureus/Vuze to cache the data more to memory if possible, also took out the option to write complete pieces to disk as soon as it's ready.
There difference is huge, same torrent in Transmission causes the write-led on the drive to keep on constantly and the download speeds are in 3-5MB/s moving up and down all the time and the drive itself is clearly struggling. Azureus/Vuze on the other hand keeps chuggin along 11MB/s almost instantly and the hard disk light goes on and off - sort of pulsing even when it's hogging data at much faster rate. This is the reason why I'm not using Transmission at the moment, IMHO it's unusable on faster connections.
Is it possible to increase the read/write cache size in OSX Transmission? Maybe through some sort of plist hack? Most of modern machines have huge amount of memory from 2-4GB so why don't we just use it. Unused memory is wasted memory.
I don't mean to sound harsh,.Transmission is an excellent piece of software. It's much lighter, prettier, easier to use than the Vuze, but when it comes to raw speed Vuze/Azureus simply beats the cr*p out of Transmission and it seems only because of the better caching options.
Sorry about the spelling mistakes + crappy grammar. English isn't my native language and also I'm typing this message at around 05:00 ZZz.zzZzzzz
.Re: Disk Cache decrease the read and write frequency of your HD
I am a little surprised when I discovery the fact that transmission dosn't support memory cache. I think this is quite a necessary and desirable feature for modern bt client. Hopefully transmission's developers can add this feature soon.
Re: Disk Cache decrease the read and write frequency of your HD
Hello,
Do you have any plans on adding this great and needed feature?
Regards,
rev
Do you have any plans on adding this great and needed feature?
Regards,
rev
Re: Disk Cache decrease the read and write frequency of your HD
Indeed, good caching is more and more imperative with faster internet-connections...
Re: Disk Cache decrease the read and write frequency of your HD
+1 for this enhancement request
Although usually all OS using some kind of file cache I too think it would be great to have it implemented, would help in case of fast connection and also in terms of hdd heat, noise, lifetime.
Using for example a solid AVL balanced tree library to store chunk data indexed by last access time could keep the most requested (seeded) piece of data in memory for fast(er) access without re-read it from hdd.
Similar approach in case of downloads to keep x number of consecutive chunks in memory then write it in one shot could speed up things and also would increase hdd performance/lifetime.
Of course on memory limited system caching could be disabled and size of cache should be configurable.
I see ticket #1521 is opened to address this but last response is 5 months old.
Thanks and regards.
Although usually all OS using some kind of file cache I too think it would be great to have it implemented, would help in case of fast connection and also in terms of hdd heat, noise, lifetime.
Using for example a solid AVL balanced tree library to store chunk data indexed by last access time could keep the most requested (seeded) piece of data in memory for fast(er) access without re-read it from hdd.
Similar approach in case of downloads to keep x number of consecutive chunks in memory then write it in one shot could speed up things and also would increase hdd performance/lifetime.
Of course on memory limited system caching could be disabled and size of cache should be configurable.
I see ticket #1521 is opened to address this but last response is 5 months old.
Thanks and regards.
Re: Disk Cache decrease the read and write frequency of your HD
I may be incompetent, but I fail to see how a user-space cache is better than using the operating system's buffer cache. Could you please explain?
The buffer cache has the following advantages:
* it shrinks automatically on memory shortage, even when the shortage is due to another application (which Transmission knows nothing about);
* the data structures used by the OS cache have been carefully tuned, way more than what the Transmission developers could do;
* going through the OS cache avoids one memory to memory copy.
Please point me to any information about how a user-space cache actually improves things.
(Note that it may be the case that you are using an operating system that happens to have a badly implemented buffer cache. In that case, please complain to your OS vendor, not to us.)
--Juliusz
The buffer cache has the following advantages:
* it shrinks automatically on memory shortage, even when the shortage is due to another application (which Transmission knows nothing about);
* the data structures used by the OS cache have been carefully tuned, way more than what the Transmission developers could do;
* going through the OS cache avoids one memory to memory copy.
Please point me to any information about how a user-space cache actually improves things.
(Note that it may be the case that you are using an operating system that happens to have a badly implemented buffer cache. In that case, please complain to your OS vendor, not to us.)
--Juliusz
Re: Disk Cache decrease the read and write frequency of your HD
Juliusz,
Actually I said something should be done as cache would improve things. If the idea of user-space caching is not welcomed I am happy to accept any other improvement. But:
1. as I see in ticket 1521 so far the only file management is done in a way of managing couple of active files, "IO caching in libtransmission is very crude. We keep a short list of the most active files and leave them open, leaving the caching to the OS." - so relying on OS caching which is too generic therefore not very effective for torrent load pattern: many small random read. My understanding of the ticket is that current transmission is io extensive. Also as per this thread users have the same experience.
2. posix_fadvise you suggesting is not cross-platform supported.
3. There should be a reason if other torrent clients implement such user-space cache.
4. Hm, complaining to OS vendor ... I can be wrong of course but OS vendors fine tune their file cache subsystem as much as they can but I guess if Ubuntu, FreeBSD and Mac users experience similar symptom using transmission then probably it might be application related?
All my gratitude to all transmission developers, great application, I like it very much - but here I think there is a valid "requirement" (admitting it sounds odd in relation to a community developed open/free sw).
Actually I said something should be done as cache would improve things. If the idea of user-space caching is not welcomed I am happy to accept any other improvement. But:
1. as I see in ticket 1521 so far the only file management is done in a way of managing couple of active files, "IO caching in libtransmission is very crude. We keep a short list of the most active files and leave them open, leaving the caching to the OS." - so relying on OS caching which is too generic therefore not very effective for torrent load pattern: many small random read. My understanding of the ticket is that current transmission is io extensive. Also as per this thread users have the same experience.
2. posix_fadvise you suggesting is not cross-platform supported.
3. There should be a reason if other torrent clients implement such user-space cache.
4. Hm, complaining to OS vendor ... I can be wrong of course but OS vendors fine tune their file cache subsystem as much as they can but I guess if Ubuntu, FreeBSD and Mac users experience similar symptom using transmission then probably it might be application related?
All my gratitude to all transmission developers, great application, I like it very much - but here I think there is a valid "requirement" (admitting it sounds odd in relation to a community developed open/free sw).
Re: Disk Cache decrease the read and write frequency of your HD
I've got a working write cache for transmission. I've been unable to measure any improvement, but then I'm using a modern OS with a modern filesystem and a very high commit time (10 minutes).
I'll tune it some more, then publish the patch; we'll see if anyone can collect any hard performance data.
I'm genuinely curious: I haven't seen any hard data on the performance benefits of a user-space disk cache in BitTorrent. If the claimed benefits actually exist, it shouldn't be hard to prove it.
--Juliusz
I'll tune it some more, then publish the patch; we'll see if anyone can collect any hard performance data.
What is that reason?radir wrote:There should be a reason if other torrent clients implement such user-space cache.
I'm genuinely curious: I haven't seen any hard data on the performance benefits of a user-space disk cache in BitTorrent. If the claimed benefits actually exist, it shouldn't be hard to prove it.
--Juliusz