I've been having a lot of problems with corrupt data for the last couple months, after years of using Transmission without this problem. A torrent I just finished downloading was 1.65 GB in size and it came with another 1.16 GB of corrupt data, according to the torrent details.
I took a look at some of the relevant source code in libtransmission: it appears that Transmission should be banning these peers fairly quickly, so either that isn't working or I'm being swarmed with bad peers/data. I suppose the latter could signify a bad network connection also?
Looks like all the relevant debug info is dumped to the message log, so I'll start looking through that from now on. Any hints on how I might troubleshoot this would be appreciated.
Lots of corrupt data
Re: Lots of corrupt data
Its unlikely that much corrupt data comes from peers. Its more likely that you have a hardware problem, either RAM or disk, even a bad network card, bad firmware or a combination of options that make it misbehave.
If TCP is used then data transmission is verified at the network level, Transmission never receives bad data unless its from malicious peers.
If UDP is used (with uTP protocol) there is no verification at the network level, the protocol should do the verification (but I don't know the details of uTP, in general its not required that a protocol does verification).
Also make several tests to try to discard "bad peers" vs protocol vs local failure. For instance, download torrents that come from servers (e.g. Linux distributions), different sources (i.e. trackers). If all have corrupt data problems, then its a hardware problem.
No.jujup wrote:I suppose the latter could signify a bad network connection also?
If TCP is used then data transmission is verified at the network level, Transmission never receives bad data unless its from malicious peers.
If UDP is used (with uTP protocol) there is no verification at the network level, the protocol should do the verification (but I don't know the details of uTP, in general its not required that a protocol does verification).
The obvious would be to disable the uTP protocol.jujup wrote:Any hints
Also make several tests to try to discard "bad peers" vs protocol vs local failure. For instance, download torrents that come from servers (e.g. Linux distributions), different sources (i.e. trackers). If all have corrupt data problems, then its a hardware problem.
Re: Lots of corrupt data
Thanks for the concrete tips. I'm not seeing as much corrupt data on my last couple torrents, maybe a couple percent now. I'll look for the stuff you mentioned and report back if it's happening again and I have a better idea what's going on.
Re: Lots of corrupt data
Alright, looked into this a bit more last night. I'm using Rene Berber's build of Transmission Qt 2.82 for Windows, been using his builds for the last couple years. I first tried downloading a couple popular torrents using my USB 3G stick, which is what I use most of the time, and was seeing 1 MB of corrupt data for every 2 MB of valid data. I tried switching to a different wifi network with different networking hardware, same problem. Downloading and running a disk utility showed no SMART errors on my SSD; ran the Windows Memory Diagnostic, no issues there either. I downloaded a 256 MB torrent for an open source OS, no corrupt data.
Finally, I went back to the USB 3G connection and the popular torrents and let them finish downloading. Over time, the amount of corrupt data went down for one torrent, perhaps because the bad peers were being banned by Transmission. On a 1 GB torrent, the checksum fails stopped when it was about 30% downloaded and it ended up with around 15% in additional corrupt data getting downloaded. Another 600 MB torrent showed no real decline in checksum fails over time and ended up with 25% additional corrupt data. Neither was as bad as the 70% additional corrupt data I saw before, but still problematic. I did not try disabling uTP, will try that at some point, but I think peers spreading corrupt data is the problem.
This additional wasted download is a problem because my main internet connection, the 3G connection, has a quota, so the wasted corrupt data directly affects how much data I can use every month.
One problem is that the default message log in the Qt interface logs checksum fails, but doesn't say which IPs are causing it. This makes it harder for the user to proactively ban bad peer IPs by adding them to a blocklist, not to mention persisting that bad peer data for the next time you open Transmission. I've never really used blocklists, looks like I'll have to start writing my own now. Looking at the Transmission source while writing this post, it appears that bad peers can be logged to stdout by setting the TR_DEBUG_FD environment variable but I'm not sure if that works on Windows. I'll try it out and see. Update: I just ran "set TR_DEBUG_FD=1" in cmd.exe and launched Transmission and it crashed on me, the same when I set it to 2. Removing the variable allowed it to run properly again when launched from the command line.
It would help if Transmission reported bad peers directly through the GUI interface, rather than having to go to more inconvenient debug logs to get this data, so that users had more control over fixing this problem.
btw, I've been using Transmission for many years now, great piece of software,
first time I've had to report a problem.
Finally, I went back to the USB 3G connection and the popular torrents and let them finish downloading. Over time, the amount of corrupt data went down for one torrent, perhaps because the bad peers were being banned by Transmission. On a 1 GB torrent, the checksum fails stopped when it was about 30% downloaded and it ended up with around 15% in additional corrupt data getting downloaded. Another 600 MB torrent showed no real decline in checksum fails over time and ended up with 25% additional corrupt data. Neither was as bad as the 70% additional corrupt data I saw before, but still problematic. I did not try disabling uTP, will try that at some point, but I think peers spreading corrupt data is the problem.
This additional wasted download is a problem because my main internet connection, the 3G connection, has a quota, so the wasted corrupt data directly affects how much data I can use every month.
One problem is that the default message log in the Qt interface logs checksum fails, but doesn't say which IPs are causing it. This makes it harder for the user to proactively ban bad peer IPs by adding them to a blocklist, not to mention persisting that bad peer data for the next time you open Transmission. I've never really used blocklists, looks like I'll have to start writing my own now. Looking at the Transmission source while writing this post, it appears that bad peers can be logged to stdout by setting the TR_DEBUG_FD environment variable but I'm not sure if that works on Windows. I'll try it out and see. Update: I just ran "set TR_DEBUG_FD=1" in cmd.exe and launched Transmission and it crashed on me, the same when I set it to 2. Removing the variable allowed it to run properly again when launched from the command line.
It would help if Transmission reported bad peers directly through the GUI interface, rather than having to go to more inconvenient debug logs to get this data, so that users had more control over fixing this problem.
btw, I've been using Transmission for many years now, great piece of software,
first time I've had to report a problem.Re: Lots of corrupt data
Sorry for the late response, turns out rb07 was right about uTP being the culprit. Disabling it fixed the excessive corruption, wanted to make sure the problem didn't recur before posting again.