Fluctuating speeds to SD-card

Ask for help and report issues with the Mac OS X version of Transmission
Post Reply
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Fluctuating speeds to SD-card

Post by dasdreHmomenT »

Hi!

Since the SSD on my Macbook Air only has a limited amount of space I bought an SD card for data and was wanting to use it as target for torrents as well.
However when using it my download (and upload) speeds fluctuate wildly, going up to several MB/s (in the range of what I get when downloading to the SSD) to only several KB/s or halting altogether for a couple of seconds. I can keep speeds down by accessing the sd-card from another program, so I suspect a IO-problem. After some research I thought about experimenting with disk cache size, but as this requires building (and I'm not that knowledgable, and would need to download XCode beforehand) I decided to ask the community first:
Do you think that cache settings are a solution here? Do you have any other ideas?

(uTorrent, which I used for testing this behaves similarly, but the fluctuations seem to be more "smooth", going slowly down to several kbs before slowly rising again. While slow, I see high disk writes, alternating between 8, 12 and 16 MB/s. I don't see this behaviour in Transmission though.)

using Transmission 2.82 (14160) on OS X 10.8.5 (12F45), Macbook Air 2012, Sandisk 64GB microSDXC Ultra

thank you very much!
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Re: Fluctuating speeds to SD-card

Post by dasdreHmomenT »

so,

I did compile with cache size set to 16, 32 and 64 mb, none of it makes a difference.
also,while I can keep speeds low by accessing the sd-card, I don't see any writing to it while the download halts.
also, upload halts too when this happens, shouldn't that at least come from cache? even from OS level disk cache, I have 2 GB free memory and loads of inactive.
monitoring my network I can see that this behaviour makes me lose about half the time I am connected.

Do you have any idea what could be causing this and how to resolve it?

thank you!
killemov
Posts: 533
Joined: Sat Jul 31, 2010 5:04 pm

Re: Fluctuating speeds to SD-card

Post by killemov »

I would advise against using flash storage for bittorrent downloads. The filesystem is usually fat/exfat and the hardware is usually connected to the USB-controller. Both are consuming CPU cycles. Did you know that you are wearing out the SD-card as well? (Some lesser brands / older cards start discarding blocks after 1000 write cycles.)
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Re: Fluctuating speeds to SD-card

Post by dasdreHmomenT »

thanks for your answer!

I get what you're saying. I've opted for the sd-card as I use my laptop very mobile, meaning it never stays in the same place for long so external hard drives, while used for archiving, don't fit the bill as a permanent storage expansion. really should've got the bigger SSD, but here I am.

processor usage doesn't bother me much, the battery in the MBA is amazing. but 1000 writes? that's like nothing! after you mentioned it I remembered reading that elsewhere too, but I find it hard to believe that they're allowed to sell stuff like this. well, maybe it will die on me before the warranty runs out… still not a very happy outlook. (you did say older/cheaper cards though, so hopefully my 2012 sandisk will last longer)

concerning the download speeds though, is there anything else I could try out? (in the meantime I have also tried to use hfs+ instead of exfat for filesystem, with no difference)
can we tell transmission to only write complete parts anyhow, as I suspect that would better suit the card's abilities (vastly better on bigger files). I get average speeds around 12 MB/s while transferring files via Finder, even if it's a folder containing many small ones average speed is at around 9 MB/s. my internet doesn't let me go over ~5 MB/s anyway, so I still hope there is a way to not let the card be the bottleneck.

Thank you all again!
blacke4dawn
Posts: 552
Joined: Sun Dec 13, 2009 10:44 pm

Re: Fluctuating speeds to SD-card

Post by blacke4dawn »

dasdreHmomenT wrote:processor usage doesn't bother me much, the battery in the MBA is amazing. but 1000 writes? that's like nothing! after you mentioned it I remembered reading that elsewhere too, but I find it hard to believe that they're allowed to sell stuff like this. well, maybe it will die on me before the warranty runs out… still not a very happy outlook. (you did say older/cheaper cards though, so hopefully my 2012 sandisk will last longer)
Those writes for the smallest possible addressable physical sector, not the card as a whole. This means that over time the effective storage capacity will shrink but the card will remain usable. Even SSD's have (or at least had, it was one of the major negative points during gen 1) this problem and "prolongs" the health of the disk by physically spreading out the writes as much as possible.
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Re: Fluctuating speeds to SD-card

Post by dasdreHmomenT »

blacke4dawn wrote:
dasdreHmomenT wrote:processor usage doesn't bother me much, the battery in the MBA is amazing. but 1000 writes? that's like nothing! after you mentioned it I remembered reading that elsewhere too, but I find it hard to believe that they're allowed to sell stuff like this. well, maybe it will die on me before the warranty runs out… still not a very happy outlook. (you did say older/cheaper cards though, so hopefully my 2012 sandisk will last longer)
Those writes for the smallest possible addressable physical sector, not the card as a whole. This means that over time the effective storage capacity will shrink but the card will remain usable. Even SSD's have (or at least had, it was one of the major negative points during gen 1) this problem and "prolongs" the health of the disk by physically spreading out the writes as much as possible.
I get that. still seems little to me, that means a professional photographer who uses the whole of his card every working day will be stuck with an unusable one after 2-3 years. but thank you for clarifying.

on topic - changing the cache to 512 mb (expectedly) makes the breaks appear less often but for a longer time. not perfect, but usable.
is there a way to tell transmission to "dynamically" write the cache to disk (say total cache 512, but every 256 are written to disk) while continuously downloading?
as the breaks are shorter than the download periods it appears to me that in a perfect world they need not appear at all (processor is definitely not the bottleneck for doing both simultaneously)

thank you!
blacke4dawn
Posts: 552
Joined: Sun Dec 13, 2009 10:44 pm

Re: Fluctuating speeds to SD-card

Post by blacke4dawn »

I took a little deeper look at the write cycle limit for SD cards and it seems that they are (maybe only higher end?) up to 100k or even 1mill write cycles before they "give up", so you shouldn't really have to worry about writing it to death anymore.

As for writing out the cache more often, I'm fairly sure you need to change the source code for that.
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Re: Fluctuating speeds to SD-card

Post by dasdreHmomenT »

ok, thank you, that's a relief!

still I have no solution for the speed drops.
notice that with a bigger cache transmission's behaviour is similar to that of utorrent described in my first post, just that it stops transferring completely while writing out the cache, utorrent only slows down.
I don't really understand how that works though. why does transmission stop downloading while writing out the cache? does it do that too when I'm downloading to the internal SSD, but the stops are so short that I don't notice (i had the standard cache of 4mb when I last tried, gotta try again with the 512-build) ? Also, having two torrents running simultaneously, one to the sd-card, one to the SSD, they both stop when data is written to the sd-card. I would understand if the processor got clogged, but it is only used a little (~12%, out of 400 for 4 cores).

In a nutshell: I don't really care how often the cache is emptied, but shouldn't it be emptied without stalling transfers? isn't that part of the reason for a cache, besides reducing disk access time?

and given that the cache of an appropriate size can be written out faster than acquired, shouldn't there be a way to not have these stops at all?


EDIT: I tried downloading to the SSD with the 512mbcache build and here transfers are continuous, not halting or slowing down when writes occur (they are "spikey" but more often than every 512 mb, also activity monitor does not show 512 mb RAM in use for transmission but significantly lower, all of which suggests that transmission does write out the cache dynamically) why can't it be like that when I'm using the sd card?


thank you all for your input!
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Re: Fluctuating speeds to SD-card

Post by dasdreHmomenT »

Another update:

since I have RAM to spare I tried with a cache of 2 gb. the result is that about 500mb, sometimes more, get downloaded before being written to disk (I can see RAM consumption rising and falling in Activity Monitor), which again stalls all in- and outgoing transfers until it is finished. but I don't see continuous writing in Activity Monitor, only in Terminal iotop. seems like the OS cache is also doing something, doesn't it? Also, this seems to take very long, longer than it should take for 500 mb to be written at rates of about 6mb/s (which is reported in iotop for transmission).

I twice (so this is repeatable) quitted transmission while it was writing, and both times it seems that, while waiting for a time and showing the "quitting" label, it didn't finish writing out the cache. When I resumed, there was less completion, "check local data" seemed to rise it again but not quite as much as was reported as downloaded before quitting.

Another thing: When Transmission writes the file to allocate disk space at the beginning of a download its GUI becomes unresponsive and I get the spinning beachball. This does not happen when the cache is written out.

So: Any ideas? Am I missing something obvious?
Are there other tips how to optimise torrenting to an SD card?

thanks!
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Re: Fluctuating speeds to SD-card

Post by dasdreHmomenT »

sure hope to be not too much of a pain, but does anyone of you know?

why does transmission stop transferring (up and down) while emptying the cache to the SD-card?
Why does it stop torrents that reside on another disk (the SSD)?
Why doesn't it do that when only writing to the SSD?
(processor usage is at about 0,7% out of 400 while doing this)

I would be grateful for solutions, but I'd also like to understand!
I would also gladly conduct further experiments if you point me in the right direction!

thank you all!
dasdreHmomenT
Posts: 8
Joined: Fri Nov 08, 2013 1:47 am

Re: Fluctuating speeds to SD-card

Post by dasdreHmomenT »

I'm sorry to be talking to myself so much.
Could anybody just point me in the right direction to read up on what is going on here, because as my last two posts tried to convey, I'm not only looking for a solution but also want to understand the principle behind my failures.
My lectures of documentation and forum threads concerning the cache led me to believe that the stalling while writing is not expected behaviour. (I also learned that the usefulness of the write cache in modern OSs is highly debated. I am still not sure whether that might have something to do with my problem. OS cache interfering?)

thank you very much!
tmssdc
Posts: 1
Joined: Sat Jun 25, 2022 7:15 pm

Re: Fluctuating speeds to SD-card

Post by tmssdc »

Having the exact same problem 9 years later... (On linux) Too bad there isn't more info on this.
Post Reply