Transmission still hangs horribly

[Jordan]

On a torrent with 390GB size and 23,273 parts I'm getting only up to 600kB/s (that's less than 10% of what it should be) and the CPU usage is maxed out all the time.

<snip>

I do compare it to the speeds I can achieve on a much slower laptop with Win7 + uT (so what you wrote doesn't make sense).

So on your benchmark system, you're seeing sustained downloads of 6 megabytes per second?

[szafran]

nope - 7,2-7,8 MB/s after a few seconds after starting the torrent.

[blacke4dawn]

On a torrent with 390GB size and 23,273 parts I'm getting only up to 600kB/s (that's less than 10% of what it should be) and the CPU usage is maxed out all the time.
I'll pull an update and do a debug dump later today.

I'm very interested to know how you are able to calculate what the speed should be considering that you nothing about the other peers upload capability and current load. Note that this is very different from the speed you have available to you for downloading data.

Read the whole topic - I do compare it to the speeds I can achieve on a much slower laptop with Win7 + uT (so what you wrote doesn't make sense). In addition the whole app chokes, every few seconds (for a couple of more seconds) it stops responding (via gtk frontend and www frontend). These chokes are at the same time that the app goes 100% CPU (the rest of the time it's about 95-98% CPU).

The point is that just because you get those speeds on one app is NO guaranty that you should achieve the same speeds in another. You can connect to other other peers which has very low or heavily overburdened upload capacity, some may actively filter out certain torrent clients and so on and so on. There is a huge amount of factors present that is outside of your control that does affect the final transfer speed.

Transmission (and the libraries it depends on) does their best to implement their standards/functionality but sometimes it doesn't get 100% compatible. This can manifest itself in that some apps freeze/slow down even when the actual error is not in that particular app but rather a library.

[szafran]

The point is that just because you get those speeds on one app is NO guaranty that you should achieve the same speeds in another. You can connect to other other peers which has very low or heavily overburdened upload capacity, some may actively filter out certain torrent clients and so on and so on. There is a huge amount of factors present that is outside of your control that does affect the final transfer speed.

Transmission (and the libraries it depends on) does their best to implement their standards/functionality but sometimes it doesn't get 100% compatible. This can manifest itself in that some apps freeze/slow down even when the actual error is not in that particular app but rather a library.

Yes, yes I know - I'm not new to this topic.
The thing is that I know to who I'm connecting and what speeds I can achieve from them (number of seeds on this torrent is much much lower than my max seeds per torrent setting, so both clients connect to the same people - and uT gets max speeds 2-3secs after starting the torrent, and Transmission after a day and 15h now still averages only at 600kB/s)

Anyways the thing is that all torrents with low part count (download total size doesn't matter, so the part size doesn't matter also) download without problems (and at max speeds) - so it's something with the code that takes care of that (and the libs are the same on low and high part count).

Today I've tried some other clients, on both machines - every other client that I've tried works fine on this torrent (on my NAS there can be sometimes IO bottlenecks when the HDDs are writing something etc., but this is very rare since all my apps use a 13x2TB RAID5 array spanned across 3 controllers, so it sustains a 600MB/s transfer that can go up to 900MB/s)

[Jordan]

Szafran, if you're trying to download a very large torrent at a sustained 7 MB/s, you may be hitting a use case that the developers haven't tested for... I don't know about livings124, but I don't have that kind of hardware / bandwidth available to me.

The gdb traces are interesting: both of them show libtransmission spending time in places that I don't usually show up when I do CPU profiling. And neither showed the usual disk IO bottlenecks (not surprising, given your hardware setup).

If you'd like to take this further, I think it would be very interesting to see what perf has to say about where transmission is spending its cycles in your use case.

Code: Select all

$ cd transmission-svn-source-code-directory/gtk/
$ perf record -g ./transmission-gtk
(let it run for awhile under heavy load so perf can record that activity, then exit Transmission)
$ perf report -g
(and pastebin that output, and post a link here)

[szafran]

Updated to svn r13827 before test.

First test with the 390GB torrent (and screens named big):

Code: Select all

szafran@NAS:~/.sources/transmission/transmission.svn$ perf record -g ./transmission-gtk
WARNING: Kernel address maps (/proc/{kallsyms,modules}) are restricted,
check /proc/sys/kernel/kptr_restrict.

Samples in kernel functions may not be resolved if a suitable vmlinux
file is not found in the buildid cache or in the vmlinux path.

Samples in kernel modules won't be resolved at all.

If some relocation was applied (e.g. kexec) symbols may be misresolved
even with a suitable vmlinux or kallsyms file.

./transmission-gtk: No such file or directory
szafran@NAS:~/.sources/transmission/transmission.svn$ cd gtk
szafran@NAS:~/.sources/transmission/transmission.svn/gtk$ perf record -g ./transmission-gtk
WARNING: Kernel address maps (/proc/{kallsyms,modules}) are restricted,
check /proc/sys/kernel/kptr_restrict.

Samples in kernel functions may not be resolved if a suitable vmlinux
file is not found in the buildid cache or in the vmlinux path.

Samples in kernel modules won't be resolved at all.

If some relocation was applied (e.g. kexec) symbols may be misresolved
even with a suitable vmlinux or kallsyms file.


(transmission-gtk:23003): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(transmission-gtk:23003): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed
[ perf record: Woken up 403 times to write data ]
[ perf record: Captured and wrote 101.212 MB perf.data (~4422018 samples) ]

Second test - 1,28GB and 307 parts (and screens named small - this one got me up to 40% CPU):

Code: Select all

szafran@NAS:~/.sources/transmission/transmission.svn/gtk$ perf record -g ./transmission-gtk
WARNING: Kernel address maps (/proc/{kallsyms,modules}) are restricted,
check /proc/sys/kernel/kptr_restrict.

Samples in kernel functions may not be resolved if a suitable vmlinux
file is not found in the buildid cache or in the vmlinux path.

Samples in kernel modules won't be resolved at all.

If some relocation was applied (e.g. kexec) symbols may be misresolved
even with a suitable vmlinux or kallsyms file.


(transmission-gtk:31902): GLib-GObject-WARNING **: invalid (NULL) pointer instance

(transmission-gtk:31902): GLib-GObject-CRITICAL **: g_signal_connect_data: assertion `G_TYPE_CHECK_INSTANCE (instance)' failed
[ perf record: Woken up 31 times to write data ]
[ perf record: Captured and wrote 8.325 MB perf.data (~363724 samples) ]

From what I see, I think that it shows exactly wheres the problem (from what I suspect those are the functions taking care of parts, but that's just speculation since I haven't looked for it in the sources).

[Jordan]

Szafran,

I'm not sure why you used screenshots for that; but regardless, it's interesting output. Obviously the thing to look at is why countArray() and countRange are being called so heavily.

Any chance you could pastebin the "perf report -g" of those to show the call chains for them?

[szafran]

those are screenshots of perf report -g (so there's no way to paste bin an app window)

[Jordan]

Maybe "perf report --call-graph", then?

The output in those screenshots looks more like what I see when I use "perf report --gtk"

[szafran]

"perf report --call-graph" gives the same thing (an app window - doesn't matter if launched under console window within gnome or on a ssh session).
I've tried "perf script", but this generates 750MB log file, so no pastebin with this one.
Will this log help (if it will then I can compress it and send you a link), or do we need to try to find other solution?

btw.

Code: Select all

szafran@NAS:~/.sources/transmission/transmission.svn/gtk$ perf --version
perf version 3.2.35

[szafran]

RAR has done pretty good job with perf.data and perf.log, so I'll put them out anyways:

snip snip

[Jordan]

Grumble. I feel like we're on the cusp of some usable information, but aren't quite there yet

So when I run that perf data through perf report, I get some pretty useless information:

# Overhead Command Shared Object Symbol

Code: Select all

# ........  ...............  .............................  ......
#
    95.77%  transmission-gt  transmission-gtk               [.] 0x0004f88a        
            |          
            |--23.53%-- 0x483d4c
            |          |          
            |          |--99.99%-- 0x4841c2
            |          |          |          
            |          |          |--99.99%-- 0x484510
            |          |          |          |          
            |          |          |          |--99.99%-- 0x484589
            |          |          |          |          0x4849dc
            |          |          |          |          |          
            |          |          |          |          |--100.00%-- 0x4872f1
            |          |          |          |          |          0x495552

So something in transmission-gtk used 95% of the CPU, and was called... from somewhere... by someone...

Szafran, if there's no way to get perf report to give you a true dump s. t. you can pastebin the first 20 or so pages, could you take another screenshot of the case where countArray() is taking up >80% of the CPU, but click on that line s.t. it expands and shows the call tree?

Thanks

[szafran]

I posted the perf.data with the log so you could get what you need - bummer it didn't work.

I've tried my best with those screens - it's really a pain that I'm unable to save it as a text file instead of showing that app window.

[szafran]

and one more...

[szafran]

So ?
What it would give me, if the speeds (not the one shown by T, but the ones measured directly on eth0) are so slow (so in that case the packet count would be much much lower than when I'm DLing with full speed) ?

I'm sure the screens are visible to you - it's cleanly visible on them that procedures that take care of parts (counting, sorting etc) are responsible for slow speeds - T must wait for those procedures to finish to continue downloading.