Transmissionbt extreme memory usage on ARM-Linux

[stulluk]

Hi,

We have cross compiled transmissionbt for our Arm-Linux system. Download is OK, Web-If is ok, everything is fine.

However, the used memory grows up quickly, and uses all of DRAM in a few minutes.

Our system have 216Mhz Arm-Cpu and 256Mbytes of DRAM ( 140Mbyte is available to userspace ).

We have limited peer amounts, cache-mb...etc and tried almost everything in settings.json file.

We also cross compiled 2.42 and 2.75 for checking.... But no luck. After a few minutes, transmission uses all of the free memory in the system, and other apps start to get killed because of "Out of memory" error.

Below is our latest tested settings.json file:

Code: Select all

{
    "alt-speed-down": 50, 
    "alt-speed-enabled": false, 
    "alt-speed-time-begin": 540, 
    "alt-speed-time-day": 127, 
    "alt-speed-time-enabled": false, 
    "alt-speed-time-end": 1020, 
    "alt-speed-up": 50, 
    "bind-address-ipv4": "0.0.0.0", 
    "bind-address-ipv6": "::", 
    "blocklist-enabled": false, 
    "blocklist-url": "http://www.example.com/blocklist", 
    "cache-size-mb": 1, 
    "dht-enabled": true, 
    "download-dir": "/mnt/usb/disk1/tbt/download", 
    "download-queue-enabled": true, 
    "download-queue-size": 5, 
    "encryption": 1, 
    "idle-seeding-limit": 30, 
    "idle-seeding-limit-enabled": false, 
    "incomplete-dir": "/mnt/usb/disk1/tbt/download", 
    "incomplete-dir-enabled": false, 
    "lpd-enabled": false, 
    "message-level": 2, 
    "open-file-limit": 5, 
    "peer-congestion-algorithm": "", 
    "peer-limit-global": 10, 
    "peer-limit-per-torrent": 10, 
    "peer-port": 51413, 
    "peer-port-random-high": 65535, 
    "peer-port-random-low": 49152, 
    "peer-port-random-on-start": false, 
    "peer-socket-tos": "default", 
    "pex-enabled": true, 
    "port-forwarding-enabled": true, 
    "preallocation": 1, 
    "prefetch-enabled": 1, 
    "queue-stalled-enabled": true, 
    "queue-stalled-minutes": 30, 
    "ratio-limit": 2, 
    "ratio-limit-enabled": false, 
    "rename-partial-files": true, 
    "rpc-authentication-required": true, 
    "rpc-bind-address": "0.0.0.0", 
    "rpc-enabled": true, 
    "rpc-password": "{33850de2564f61c7e602b343f859ad06a662ea32ar1jVaqL", 
    "rpc-port": 9091, 
    "rpc-url": "/transmission/", 
    "rpc-username": "chipbox", 
    "rpc-whitelist": "127.0.0.1", 
    "rpc-whitelist-enabled": false, 
    "scrape-paused-torrents-enabled": true, 
    "script-torrent-done-enabled": false, 
    "script-torrent-done-filename": "", 
    "seed-queue-enabled": false, 
    "seed-queue-size": 10, 
    "speed-limit-down": 100, 
    "speed-limit-down-enabled": false, 
    "speed-limit-up": 100, 
    "speed-limit-up-enabled": false, 
    "start-added-torrents": true, 
    "trash-original-torrent-files": false, 
    "umask": 18, 
    "upload-slots-per-torrent": 14, 
    "utp-enabled": true
}

Could you please let us know, how we can limit memory usage of transmission-bt to 50Mbytes ( for example) ?

Best Regards

[Jordan]

I'm not seeing anything like that kind of use. How many torrents do you have loaded?

I just ran a valgrind massif session with transmission-daemon to download a Linux iso, and also seeding a handful of torrents I already had loaded, and massif reports a peak use of 11.8 MiB.

Do you have any other information on how I can reproduce in testing the behavior that you're seeing?

[stulluk]

I'm not seeing anything like that kind of use. How many torrents do you have loaded?

I just ran a valgrind massif session with transmission-daemon to download a Linux iso, and also seeding a handful of torrents I already had loaded, and massif reports a peak use of 11.8 MiB.

Do you have any other information on how I can reproduce in testing the behavior that you're seeing?

Hi, on our box, we can easily reproduce the issue when we add 4-5 torrents by using webif.

After that, we have COM, telnet and SSH access to our box, then we call "free" command.

After 5 minutes, transmissionbt start to "eat" the memory, and no memory remains for other processes, and other processes start to get killed and by giving "Out of Memory" error.

We love transmissionbt, so we want to solve this issue, and use it on our box.

If you want, we can provide TELNET / SSH access to you remotely, so you can see what happens in realtime.

Would you like to see?

Or if you want, we can provide Strace -f result to you also.

Which one you prefer?

[Jordan]

Sure, thanks for the offer.

Mail me a throwaway ssh account name + password, and the exact steps to reproduce the behavior on your system with some public domain torrents, to [my name] @transmissionbt.com, and I'll take a look.

Also if you could ensure that valgrind and gdb are installed, and that your cross-compile of transimssion-daemon still has debugging symbols built in, that would be a help.

[stulluk]

Sure, thanks for the offer.

Mail me a throwaway ssh account name + password, and the exact steps to reproduce the behavior on your system with some public domain torrents, to [my name] @transmissionbt.com, and I'll take a look.

Also if you could ensure that valgrind and gdb are installed, and that your cross-compile of transimssion-daemon still has debugging symbols built in, that would be a help.

Hi Mr Jordan;

I just sent one e-mail to jordan (at) transmissionbt (dot) com . Please let me know if you didn't receive it.

And I prepared one STB for you, and ready for your connection. I put gdbserver executable to /tmp folder, also waiting for your signal.

I also provide my skype, MSN... contact information; please let me know when you are ready..

Waiting for your signal.

Thank you very very much again.

[fbs777]

im having the same problem here using the transmission in a router.

Details:
router: TP-Link WR1043ND (400MHZ, 32MB)
SO/firmware: OpenWrt Attitude Adjustment 12.09-rc1 / LuCI 0.11 Branch (0.11+svn9425) kernel 3.3.8
transmission version: transmission-cli 2.71-1, transmission-daemon 2.71-1, transmission-web 2.71-1 (packages from this source: http://downloads.openwrt.org/attitude_a ... /packages/ )

After start, transmission always consumes 70% of ram, than after a few minutes i need to reset the router because all the other processes is down.

In the older openwrt 10.03.1 w/ an older version of transmission (2.32-2) i was forced to reset the router just after many hours, but after the update i need to reset after a few minutes after transmission starts...

result of top and free commands without transmission start:

Code: Select all

Mem: 27376K used, 1844K free, 0K shrd, 8128K buff, 6236K cached
CPU:   0% usr   0% sys   0% nic  99% idle   0% io   0% irq   0% sirq
Load average: 0.00 0.01 0.05 2/47 22532
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
22522 22517 root     R     1500   5%   0% top
22516  1793 root     S     1220   4%   0% /usr/sbin/dropbear -P /var/run/dropbe
22482  1867 root     S     3492  12%   0% /usr/sbin/smbd -D
 1869     1 root     S     3196  11%   0% /usr/sbin/nmbd -D
 1867     1 root     S     3116  11%   0% /usr/sbin/smbd -D
22532  1867 root     S     3116  11%   0% /usr/sbin/smbd -D
  706     1 root     S     1524   5%   0% /sbin/netifd
  623     1 root     S     1508   5%   0% /sbin/syslogd -l 8 -C16
  750   706 root     S     1508   5%   0% udhcpc -p /var/run/udhcpc-eth0.2.pid
    1     0 root     S     1504   5%   0% init
22517 22516 root     S     1504   5%   0% -ash
  504     1 root     S     1504   5%   0% init
 1596     1 root     S     1500   5%   0% /sbin/watchdog -t 5 /dev/watchdog
 1882     1 root     S     1500   5%   0% /usr/sbin/ntpd -n -p 0.openwrt.pool.n
  625     1 root     S     1492   5%   0% /sbin/klogd
 1105     1 root     S     1428   5%   0% hostapd -P /var/run/wifi-phy0.pid -B
 1793     1 root     S     1152   4%   0% /usr/sbin/dropbear -P /var/run/dropbe
 1811     1 root     S     1148   4%   0% /usr/sbin/uhttpd -f -h /www -r OpenWr
 1845     1 nobody   S      964   3%   0% /usr/sbin/dnsmasq -C /var/etc/dnsmasq
  633     1 root     S      872   3%   0% /sbin/ubusd

Code: Select all

root@OpenWrt:~# free
             total         used         free       shared      buffers
Mem:         29220        27504         1716            0         8128
-/+ buffers:              19376         9844
Swap:      1048572            0      1048572

result of top and free commands after start transmission (at this point internet still work):

Code: Select all

Mem: 28088K used, 1132K free, 0K shrd, 396K buff, 4596K cached
CPU:  37% usr  23% sys   0% nic  24% idle  11% io   0% irq   2% sirq
Load average: 1.70 0.64 0.27 2/53 23528
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
23514     1 root     S    20544  70%  50% /usr/bin/transmission-daemon -g /mnt/
    5     2 root     SW       0   0%   6% [kworker/u:0]
  477     2 root     DW       0   0%   3% [usb-storage]
23329 22517 root     R     1500   5%   2% top
  250     2 root     SW       0   0%   1% [kworker/0:1]
    3     2 root     SW       0   0%   0% [ksoftirqd/0]
   95     2 root     SW       0   0%   0% [kswapd0]
  194     2 root     SW       0   0%   0% [mtdblock2]
  696     2 root     SW       0   0%   0% [jbd2/sda1-8]
 1869     1 root     S     3216  11%   0% /usr/sbin/nmbd -D
 1867     1 root     S     3136  11%   0% /usr/sbin/smbd -D
23526  1867 root     S     3136  11%   0% /usr/sbin/smbd -D
23528  1867 root     S     3136  11%   0% /usr/sbin/smbd -D
23527  1867 root     S     3136  11%   0% /usr/sbin/smbd -D
  706     1 root     S     1524   5%   0% /sbin/netifd
  623     1 root     S     1508   5%   0% /sbin/syslogd -l 8 -C16
  750   706 root     S     1508   5%   0% udhcpc -p /var/run/udhcpc-eth0.2.pid
22517 22516 root     S     1508   5%   0% -ash
    1     0 root     S     1504   5%   0% init
  504     1 root     S     1504   5%   0% init

Code: Select all

root@OpenWrt:~# free
             total         used         free       shared      buffers
Mem:         29220        28260          960            0         1068
-/+ buffers:              27192         2028
Swap:      1048572         6156      1042416

result of top command after few minutes of transmission running (at this time internet dont work anymore):

Code: Select all

Mem: 27468K used, 1752K free, 0K shrd, 1428K buff, 5368K cached
CPU:   1% usr   0% sys   0% nic  98% idle   0% io   0% irq   0% sirq
Load average: 0.33 0.97 0.76 1/53 2585
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
 2072     1 root     S    20532  70%   1% /usr/bin/transmission-daemon -g /mnt/
 2580  2575 root     R     1500   5%   1% top
 1869     1 root     S     3196  11%   0% /usr/sbin/nmbd -D
 1867     1 root     S     3108  11%   0% /usr/sbin/smbd -D
 2583  1867 root     S     3108  11%   0% /usr/sbin/smbd -D
 2581  1867 root     S     3108  11%   0% /usr/sbin/smbd -D
 2582  1867 root     S     3108  11%   0% /usr/sbin/smbd -D
  729     1 root     S     1524   5%   0% /sbin/netifd

When i choose to not start transmission in boot the router works perfect online for days.

[moeburn]

im having the same problem here using the transmission in a router.

Details:
router: TP-Link WR1043ND (400MHZ, 32MB)
SO/firmware: OpenWrt Attitude Adjustment 12.09-rc1 / LuCI 0.11 Branch (0.11+svn9425) kernel 3.3.8
transmission version: transmission-cli 2.71-1, transmission-daemon 2.71-1, transmission-web 2.71-1 (packages from this source: http://downloads.openwrt.org/attitude_a ... /packages/ )

After start, transmission always consumes 70% of ram, than after a few minutes i need to reset the router because all the other processes is down.

In the older openwrt 10.03.1 w/ an older version of transmission (2.32-2) i was forced to reset the router just after many hours, but after the update i need to reset after a few minutes after transmission starts...

result of top and free commands without transmission start:

Code: Select all

Mem: 27376K used, 1844K free, 0K shrd, 8128K buff, 6236K cached
CPU:   0% usr   0% sys   0% nic  99% idle   0% io   0% irq   0% sirq
Load average: 0.00 0.01 0.05 2/47 22532
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
22522 22517 root     R     1500   5%   0% top
22516  1793 root     S     1220   4%   0% /usr/sbin/dropbear -P /var/run/dropbe
22482  1867 root     S     3492  12%   0% /usr/sbin/smbd -D
 1869     1 root     S     3196  11%   0% /usr/sbin/nmbd -D
 1867     1 root     S     3116  11%   0% /usr/sbin/smbd -D
22532  1867 root     S     3116  11%   0% /usr/sbin/smbd -D
  706     1 root     S     1524   5%   0% /sbin/netifd
  623     1 root     S     1508   5%   0% /sbin/syslogd -l 8 -C16
  750   706 root     S     1508   5%   0% udhcpc -p /var/run/udhcpc-eth0.2.pid
    1     0 root     S     1504   5%   0% init
22517 22516 root     S     1504   5%   0% -ash
  504     1 root     S     1504   5%   0% init
 1596     1 root     S     1500   5%   0% /sbin/watchdog -t 5 /dev/watchdog
 1882     1 root     S     1500   5%   0% /usr/sbin/ntpd -n -p 0.openwrt.pool.n
  625     1 root     S     1492   5%   0% /sbin/klogd
 1105     1 root     S     1428   5%   0% hostapd -P /var/run/wifi-phy0.pid -B
 1793     1 root     S     1152   4%   0% /usr/sbin/dropbear -P /var/run/dropbe
 1811     1 root     S     1148   4%   0% /usr/sbin/uhttpd -f -h /www -r OpenWr
 1845     1 nobody   S      964   3%   0% /usr/sbin/dnsmasq -C /var/etc/dnsmasq
  633     1 root     S      872   3%   0% /sbin/ubusd

Code: Select all

root@OpenWrt:~# free
             total         used         free       shared      buffers
Mem:         29220        27504         1716            0         8128
-/+ buffers:              19376         9844
Swap:      1048572            0      1048572

result of top and free commands after start transmission (at this point internet still work):

Code: Select all

Mem: 28088K used, 1132K free, 0K shrd, 396K buff, 4596K cached
CPU:  37% usr  23% sys   0% nic  24% idle  11% io   0% irq   2% sirq
Load average: 1.70 0.64 0.27 2/53 23528
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
23514     1 root     S    20544  70%  50% /usr/bin/transmission-daemon -g /mnt/
    5     2 root     SW       0   0%   6% [kworker/u:0]
  477     2 root     DW       0   0%   3% [usb-storage]
23329 22517 root     R     1500   5%   2% top
  250     2 root     SW       0   0%   1% [kworker/0:1]
    3     2 root     SW       0   0%   0% [ksoftirqd/0]
   95     2 root     SW       0   0%   0% [kswapd0]
  194     2 root     SW       0   0%   0% [mtdblock2]
  696     2 root     SW       0   0%   0% [jbd2/sda1-8]
 1869     1 root     S     3216  11%   0% /usr/sbin/nmbd -D
 1867     1 root     S     3136  11%   0% /usr/sbin/smbd -D
23526  1867 root     S     3136  11%   0% /usr/sbin/smbd -D
23528  1867 root     S     3136  11%   0% /usr/sbin/smbd -D
23527  1867 root     S     3136  11%   0% /usr/sbin/smbd -D
  706     1 root     S     1524   5%   0% /sbin/netifd
  623     1 root     S     1508   5%   0% /sbin/syslogd -l 8 -C16
  750   706 root     S     1508   5%   0% udhcpc -p /var/run/udhcpc-eth0.2.pid
22517 22516 root     S     1508   5%   0% -ash
    1     0 root     S     1504   5%   0% init
  504     1 root     S     1504   5%   0% init

Code: Select all

root@OpenWrt:~# free
             total         used         free       shared      buffers
Mem:         29220        28260          960            0         1068
-/+ buffers:              27192         2028
Swap:      1048572         6156      1042416

result of top command after few minutes of transmission running (at this time internet dont work anymore):

Code: Select all

Mem: 27468K used, 1752K free, 0K shrd, 1428K buff, 5368K cached
CPU:   1% usr   0% sys   0% nic  98% idle   0% io   0% irq   0% sirq
Load average: 0.33 0.97 0.76 1/53 2585
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
 2072     1 root     S    20532  70%   1% /usr/bin/transmission-daemon -g /mnt/
 2580  2575 root     R     1500   5%   1% top
 1869     1 root     S     3196  11%   0% /usr/sbin/nmbd -D
 1867     1 root     S     3108  11%   0% /usr/sbin/smbd -D
 2583  1867 root     S     3108  11%   0% /usr/sbin/smbd -D
 2581  1867 root     S     3108  11%   0% /usr/sbin/smbd -D
 2582  1867 root     S     3108  11%   0% /usr/sbin/smbd -D
  729     1 root     S     1524   5%   0% /sbin/netifd

When i choose to not start transmission in boot the router works perfect online for days.

2 things: 1: "Free MEM" and "Used MEM" are not exactly accurate, because I'll bet you at least half of that ram is cached. That means it's not really "used", its just using available memory space until some other program REALLY needs the memory. Take android, for example. When you close a program, it remains in memory, because it is faster and less battery-intensive to load from memory than to load from flash. But if some other, running foreground program really needs that memory, the closed program is unloaded and the memory is freed up, all in an instant. You need to subtract "cached mem" from "used mem" to get "actual used mem".

2. 32mb is not really enough to run Transmission, and your network, all on its own. You either need A) a router with 64mb of memory (I'm using a WNDR3700v2 w/64mb, and it can run Transmission just fine), or B) a SWAP partition/file. I'm assuming you have attached a USB HDD to your router. You can either make a linux-swap partition at the BEGINNING of the drive (beginning is faster in HDD) and mkswap it, or google how to use a swap file on your existing partition. Or, for even faster performance (although I can't imagine you'd ever need it), you can use a powered USB hub, and run the SWAP partition off a USB flash drive while the torrents save to a USB HDD. Flash drives are better for SWAP partitions because they have better random access times, and SWAP reads/writes are almost always random, never sequential.

[carucio]

Hi all, I'm using Transmission 2.84 (the daemon) on an Asus RT-AC68U (ARM) router using the latest build of DD-WRT (specifically ftp://ftp.dd-wrt.com/betas/2014/10-29-2 ... rmware.trx).
The configuration is very conservative in terms of memory allowed to the application (4MB cache size, 200 peers total limit) but when adding even a single torrent to the download queue the OS's free memory quickly runs out (over 100MB get used in a matter of seconds), often causing Transmission and other processes to run out of memory and crash.
This old thread seems to describe the same exact issue so I'm wondering if anybody has found a solution by now.
Thanks for your help!
Sergio

[moeburn]

Wow, 100mb in a few seconds? That's not right, sounds like a memory leak of some sort. Mine never uses that much, and I only have 64mb with a swap partition. Try uninstalling and reinstalling transmission? Maybe your install got corrupted.

[carucio]

Thanks for confirming this it not right; last time I tried I had almost 200MB free RAM while the queue was empty.. after adding a single Torrent and waiting for a couple of minutes.. I was down to just over 40MB of free memory.
The client I'm using is the one built in dd-wrt so I cannot easily change it; I'm investigating on the WRT forums as well so.. do you think this is a WRT specific issue or a general issue with Transmission on ARM CPUs?