Use mmap instead of userspace buffers

[lvella]

So, how hard would be to put a switch on ./configure to replace all malloc'ed up/download buffers with mmap'ed pointers?

This is the idea: instead of malloc'ing a buffer, downloading into it, hash cheching, then writing to a file, I propose, for systems that support the system call mmap, to: mmap the chunk to its belonged place on a file, download into there, hash check, if cool, simply unmap. The same idea applies for upload: mmap the file chunk to a pointer, pass this same pointer to the write system call that sends the bytes to the socket, and when not needed anymore, unmap it.

This arguably leads to a better usage of system memory, and I am specially concerned about it when running transmission on a 64mb RAM machine. See, it is inevitable that the kernel will pass every disk access through the buffer cache, thus every read and write operation will involve a userspace buffer and a kernel space buffer before getting to the file, for a large number of disk pages being accessed simultaneously, we have a big usage of buffer cache and a big usage of userspace memory, to copy from/to there with read/write system calls. With mmap, we still have the kernel memory usage, but the userspace buffers are no longer needed, so we end up with half memory usage (ok, may not be exactly that) and it is up to the kernel, with it's long time studied cache replacing algorithms, to manage all the memory used by the torrent client.

rTorrent people also claims that this technique improves performance: without mmap, you need one "read" from file to userspace, and one "write" from userspace to network stack. This leads to two copies and possibly one disk access. With mmap, you only need one "write", from a virtual pointer to the network stack, and once in kernel level, the data may already be available in memory, and it is just copied once, or it is read from the file before being copied to the network stack, one copy and possibly one: one copy and one possible disk access.

[jch]

This is exactly what we do in Hekate. It simplifies the code quite a bit. Here's a screenshot of Hekate running on a 32 MB machine — ps thinks Hekate is using 4300% of available RAM.

Now mmapping the files rather than reading them in has a number of consequences. One is that the memory is populated by page faults rather than by explicit read/write system calls; depending on your hardware and your OS, this might be slower or faster. My gut instinct is that mmapping is probably slightly faster than explicit I/O for read-only mappings (as in Hekate), but perhaps not for read/write mappings (as would need to be done in Transmission). If you have any benchmark results, I'd like to see them.

The other consequence is that the amount of virtual memory you have is equal to the total amount of data you're sharing (hence the 4300% figure above). This is fine on 64-bit arches, but means that on 32-bit arches you're limited to 2 GB of data. That's fine for Hekate (we simply require 64-bit hardware), but certainly not for Transmission.

All in all, there are some interesting tradeoffs in doing what you suggest, and I'd definitely be interested in learning more on the subject. Please feel free to

--jch