Corrupt files, did not know and verifying

Ask for help and report issues not specific to either the Mac OS X or GTK+ versions of Transmission
Post Reply
SandJ
Posts: 7
Joined: Sun Mar 30, 2014 10:48 am

Corrupt files, did not know and verifying

Post by SandJ »

I realised a file I had downloaded some time ago still had an upload ratio of 0.0 and I was trying to work out why. At one point I looked in Properties and saw I only had 80-something% of the file available despite it saying 100% on the front screen. So I verified the file it and the proportion available went up. Verified it again and got the popup box saying it was downloaded.

So I checked the properties of the other files I had in Transmission and find of the 23 files I have, another 5 were in the same state: marked as 100% but the properties said I did not have 100%. And I am referring to the "Have:" information, not the "Downloaded:" information. Verifying them resulted in the last few % of errors (or whatever the wording was) disappearing and a corresponding "Torrent Complete" popup.

The only thing I use torrent for are Linux distribution .iso files. Among those with errors were one from April 2012 and, most annoyingly, one from April 2010. I have been cutting disks and bootable USB sticks from these ISOs which Transmission said were 100% complete and getting poor success at booting from them. I have wondered why some distros don't work when they should; I had come to the conclusion Linux distros are just generally unreliable. Never getting the 64 bit version of Ubuntu 10.04 working was a particular annoyance and I recently struggled with a 64-bit Linux Mint for some ages. I think I may have wasted many hours - certainly evenings and probably entire weekends - as a consequence of this.

What is the point of Transmission reporting a file is downloaded and at 100% when parts of it are corrupt or missing? Is this a failing of the torrent concept or of Transmission or something else?



BTW, the Captcha for registration is a right royal pain in the arse. Do you really need so many PC / food items in that list? And having the too-many-logins automatically enabled is also annoying. And Cupertino? Is that even real? Ha ha bloody ha. How about MugOfEspresso or BaconSarnieTown? You're supposed to make it hard for bots, not for people who are registering for support and are already annoyed.
SandJ
Posts: 7
Joined: Sun Mar 30, 2014 10:48 am

Re: Corrupt files, did not know and verifying

Post by SandJ »

BTW, it is Transmission 2.82 (14160) on Xubuntu 13.04 and these files were downloaded between 04/11/2012 and the last few weeks. When the files were downloaded this PC may have been running Xubuntu 12.04, 13.04 or 13.10 according to whatever state of upgrade it was at at the time.
x190
Posts: 5094
Joined: Sun Nov 30, 2008 4:59 am

Re: Corrupt files, did not know and verifying

Post by x190 »

Here is a patch that you could try:

https://trac.transmissionbt.com/ticket/4649#comment:39

Also, don't linux isos have checksums available that can be tested?
SandJ
Posts: 7
Joined: Sun Mar 30, 2014 10:48 am

Re: Corrupt files, did not know and verifying

Post by SandJ »

x190 wrote:Here is a patch that you could try: https://trac.transmissionbt.com/ticket/4649#comment:39
Ah. That made interesting reading. So it's not just me then.
x190 wrote:Also, don't linux isos have checksums available that can be tested?
They do, but I had assumed (yes, I am old enough to know better) I had assumed that a piece of software written for downloading stuff in pieces and stitching them together might have been relied upon to do just that. Especially since it is provided as the default BitTorrent client in at least one distro. I intentionally started using Transmission specifically for Linux .isos in the belief that it was a superior and reliable alternative to FTP.

I don't suppose this will go down very well, but I should like to express my disappointment that it should report 100% in the compact view and transfer a file to the Complete folder, when said file is not complete.
x190
Posts: 5094
Joined: Sun Nov 30, 2008 4:59 am

Re: Corrupt files, did not know and verifying

Post by x190 »

I don't suppose this will go down very well, but I should like to express my disappointment that it should report 100% in the compact view and transfer a file to the Complete folder, when said file is not complete.
The problem is that no-one can pinpoint the source of the alleged problem. Even the individual who wrote that patch has never experienced the issue. It has been suggested that faulty hardware or opening the file in certain applications may be a contributing factor.
So I checked the properties of the other files I had in Transmission and find of the 23 files I have, another 5 were in the same state: marked as 100% but the properties said I did not have 100%.
Can you elaborate on this? Are these multifile torrents with 1 or more files (originally) unchecked?
SandJ
Posts: 7
Joined: Sun Mar 30, 2014 10:48 am

Re: Corrupt files, did not know and verifying

Post by SandJ »

x190 wrote:
So I checked the properties of the other files I had in Transmission and find of the 23 files I have, another 5 were in the same state: marked as 100% but the properties said I did not have 100%.
Can you elaborate on this? Are these multifile torrents with 1 or more files (originally) unchecked?
When I upgrade on this machine, I copy the partition and upgrade the copy. The old partition gets left until I need the space. That means I can do a binary file compare between the downloaded files from back then with now.

Comparing the ~/Downloads/Torrents/Complete directory in my current Xubuntu 13.10 partition with the still existing Xubuntu 12.04 partition, I can see some differences in some of the files. Bearing in mind the Xubuntu 13.10 files have all been through the verify process and been amended:

~/Downloads/Torrents/Complete/kubuntu-13.04-desktop-amd64.iso
From 0A3C0000 to 0A414FFF is all zeroes in the older file.
From 16480000 to 164D4FFF is all zeroes in the older file.
From 25080000 to 251EEFFF is all zeroes in the older file.
From 39900000 to 39A43FFF is all zeroes in the older file.
From 39F00000 to 39F23AAF is all zeroes in the older file.
After that it gets messy as the newer file is mostly zeroes anyway with occasional strings in it, whereas the older file is all zeroes, but we are right near the end of the file anyway.

~/Downloads/Torrents/Complete/linuxmint-14.1-mate-dvd-64bit.iso
From 10D00000 to 11FFEFFF is all zeroes in the older file.
From 16D80000 to 16FFEFFF is all zeroes in the older file.
From 2B4C0000 to 2BF8BFFF is all zeroes in the older file.
Then match up to the end at 3C7FFFFF

~/Downloads/Torrents/Complete/ubuntu-12.04.2-alternate-amd64.iso
From 17F00000 to 17FFEFFF is all zeroes in the older file.
Then match up to the end at 2C0FFFFF

~/Downloads/Torrents/Complete/ubuntu-13.04-desktop-amd64.iso
From 06280000 to 06E7CFFF is all zeroes in the older file.
From 12380000 to 13FFEFFF is all zeroes in the older file.
From 1E280000 to 1E2B6FFF is all zeroes in the older file.
From 29580000 to 297FEFFF is all zeroes in the older file.
Then match up to the end at 310FFFFF

(Blimey. That's enough. That was boring.)

What I can see is that:
- two .iso files downloaded before 18/02/2013 12:19:40 were OK
- seven .iso files downloaded between 18/02/2013 12:23:11 and 07/08/2013 11:01:19 are all partly blocks of zeroes
- five .iso files downloaded since 10/10/2013 12:53:13 are OK.

However, when I ran the verify yesterday, I can see there are another 6 .iso files I have downloaded since October 2013 that were all fixed by the verify because they have a time stamp for that period of 3 hours when I ran the verify process. Those files were downloaded since the start of March 2014.

In the morning I'll boot up into the old Xubuntu 12.04 partition and see what Transmission says about the state of the files.
x190
Posts: 5094
Joined: Sun Nov 30, 2008 4:59 am

Re: Corrupt files, did not know and verifying

Post by x190 »

Are you trying to download isos which may be modifying pre-existing files and while perhaps also seeding from another torrent? If you are doing something like that, then you may be affected by this, which some of us believe may be a bad commit.
SandJ
Posts: 7
Joined: Sun Mar 30, 2014 10:48 am

Re: Corrupt files, did not know and verifying

Post by SandJ »

x190 wrote:Are you trying to download isos which may be modifying pre-existing files and while perhaps also seeding from another torrent?
Unless I misunderstand you, I don't think so. I download an iso and then it just sits there indefinitely. I think twice I've gone to download an iso I already have and I just get a message like "Torrent already exists" and nothing is downloaded.
SandJ
Posts: 7
Joined: Sun Mar 30, 2014 10:48 am

Re: Corrupt files, did not know and verifying

Post by SandJ »

SandJ wrote:In the morning I'll boot up into the old Xubuntu 12.04 partition and see what Transmission says about the state of the files.
It was showing files as 100%. Here is one of them:

Properties of an iso in X12.04: Image
Properties of same iso in X13.10 (which has been verified and fixed, hence extra content has been downloaded): Image
Last edited by SandJ on Tue Apr 01, 2014 3:18 pm, edited 1 time in total.
x190
Posts: 5094
Joined: Sun Nov 30, 2008 4:59 am

Re: Corrupt files, did not know and verifying

Post by x190 »

So 16 pieces (8 MB) got corrupted in some way during the copy process? How do you know for sure the file had undiscovered bad pieces immediately after download to 100%, and while still on the original partition and at the original location? When you tried to re-verify the file, did you do so using the original torrent file? Did a .resume file come into play? Even changed mtimes could be a factor.

If you feel some form of trac ticket #4649 is a good idea as a user option, then please add your voice there.
SandJ
Posts: 7
Joined: Sun Mar 30, 2014 10:48 am

Re: Corrupt files, did not know and verifying

Post by SandJ »

x190 wrote:How do you know for sure the file had undiscovered bad pieces immediately after download to 100%, and while still on the original partition and at the original location?
It is still in its original 12.04 partition and location. I copied the partition it is on to create a newer partition to upgrade to 13.04. The same blocks of zeroes exist on the 13.04 partition that I copied to make the current 13.10 partition, which is the one where I ran the verify.
x190 wrote:When you tried to re-verify the file, did you do so using the original torrent file? Did a .resume file come into play? Even changed mtimes could be a factor.
I'll need to test that on the 12.04 partition.
x190 wrote:If you feel some form of trac ticket #4649 is a good idea as a user option, then please add your voice there.
Since I started on this exercise because of a file I had downloaded within the last few weeks which, when verified, downloaded more bits, then I reckon the problem occurred for me in 2012, 2013 and in 2014 on 3 different partitions (yes the same disk, but I've had no other signs of any disk corruption such as not in my 1.8 Gb email system which gets written to daily and has also been dragged from partition to partition). I reckon this problem has existed for some time and still happening and there seems to be one common factor.

What I will do one night is set up shedloads of downloads and see what happens.

I'll add to the trac system, thank you for your help.
x190
Posts: 5094
Joined: Sun Nov 30, 2008 4:59 am

Re: Corrupt files, did not know and verifying

Post by x190 »

thank you for your help.
You're most welcome. I do find it strange that your original d/l is showing 4.72 MB (8 pieces) as corrupt. It's odd that anyone would send corrupted pieces for a linux iso. Are you getting your torrents from a known reliable source?

You mentioned issues with seeding. When Transmission finds a bad piece during upload, then it will pause the transfer and ask you to re-verify the file.

Unfortunately, since the developers seem to have very limited time to spend on the project recently and don't appear to be experiencing the issue themselves, then some type of auto re-verification may be the best solution.
Post Reply