Favor finishing files over beginning all files per torrent

[tuxolero]

Hi,

When I open a torrent with multiple files, it looks like transmission starts downloading all contained files in parallel. This means I have no usable file until almost everything is finished.
I think it would be better to put all download effort to one file, then the next, and so on. Only when a file is "blocked", i.e. no peer available that seeds missing blocks, start another one.

I am not sure if it is possible to do a distributed download of one file from multiple peers. If not, it is OK to download multiple files at once. This way, you have the least possible waiting time for the next file to complete in all situations.

My proposal works a bit like the task scheduler of an OS where the file with the least missing data should have the highest priority. Least missing data, because it prefers smaller files when there are multiple files with same percentage.

The scheduling between multiple torrents should not be affected by my proposal. I think it is very good as it is.

What do you think about this ?

Regards,
tuxolero

[blacke4dawn]

Seems you are misunderstanding how torrents work as a whole. They do not work on a per-file basis but rather it chops up all the data in fixed size blocks (called pieces), and these pieces can have parts (or whole) of two or more files in them. This make is possible to download the data for one single file from multiple peers, which is effectively the entire purpose behind the torrent protocol.

On a technical level you are correct in that it downloads all files in parallel since it doesn't treat them as individual files but rather a large heap of pieces, where (by default) non is more important than any other.

Second major issue is that you are effectively at the mercy of the peers you are downloading from since it is they who decide what to send to you. The most you can do to affect this is to request specific pieces and hope that the request is honored. Most torrent-clients try to keep the swarm healthy by sending out or requesting the pieces who are shared and/or available the least first so as to try and keep all pieces on the same "availability level". This means that the "effort" of downloading is not so much on the receivers end as it is on the senders end.

[tuxolero]

That's interesting. Indeed, I don't kow very much about torrent internals. I only see what I see

But if it's only a bunch of pieces, how does it work to select which files are downloaded and which are not ? When you open the properties of a torrent (at least in transmission-gtk), there is a "files" tab where you can uncheck single files you don't want to download.

[blacke4dawn]

It selects all pieces that has some data from the file wanted, which means you will get parts of the "bordering" files or even the whole of them if they are small enough. That is the difference between what is presented to the swarm (rest of the clients sharing in the same torrent) and what is presented to you as the user. On a technical level you can think of the files as a "group" of pieces where all can be (de)selected at once for convenience.

[tuxolero]

So, the client has knowledge about the relation of files and blocks. Therefore, it is possible to favor all the blocks that belong to the file that has least blocks left to download. The borders don't bother, as in most cases the torrent will be downloaded completely.
As you wrote, the client can try to request specific blocks from the peer.

This is all that's needed for making the priority handling I suggested.

Regards,
tuxolero

[tuxolero]

Yes, but this way, I would need to adjust the priorities of the remaining files regularly.
The reason is that priorities shift when a lower prio file overtakes (=gets more progress) a higher prio file when there is currently no peer that seeds the higher prio one.
I'm talking about torrents with > 10 files.

[tuxolero]

I've just tried adjusting priorities manually. The torrent has 78 small files (jpeg's). After the first 12 files were started downloading, I've set them to priority "High" and left the other ones at "Normal".
But instead of completing these 12 fles, I currently have 49 files between 12% and 63% completed, but none at 100%.

At the moment of writing this, the only peer I had for this torrent is offline, so I have nothing useful...
With my proposal, I would have had at least a few of the photos and I could decide if I continue downloading this torrent or just remove it from transmission.

When the peer was online, I've verified that the peer has 100% of the torrent available ("Peers" tab of torrent properties). So this can not be the outcome of "what wasn't downloaded wasn't available".

I have two questions about this:

1)
Does setting the priority have any effect at all ? Currently, it appears futile to me.

2)
Who decides which blocks are trasmitted ? Seeder or Leecher ?

I think my idea would not harm the concept of distributing the load by sending different blocks to different leechers. The seeder could just send blocks of different files to the different leechers.

[tuxolero]

OK, I think I need to ask question 2 a bit more precisely ...
Let's assume the seeder has 100% available and some bandwidth left for me, is there a way to request some specific blocks or do I have to take what I get ? Or does it depend on the peer's software ?

About priority setting ... This is not the first time I was trying to set priorities without any notable effect. In other torrents with only a few but bigger files (also verified that the peer had 100% available), I have set one file to "High" and all the others to "Low". But still, the "High" file was completed as last one ... that's how I got the impression that priority setting does not work very well.