ANNOUNCE: Preliminary µTP support for Transmission

Discussion of Transmission that doesn't fit in the other categories
Post Reply
jch
Posts: 175
Joined: Wed May 13, 2009 12:08 am

ANNOUNCE: Preliminary µTP support for Transmission

Post by jch »

As some of you may have noticed, Jordan and I have recently been working on µTP support for Transmission. This work is being sponsored by BitTorrent, Inc. We are very grateful for this opportunity to do some low-level hacking on Transmission, something we both enjoy.

0. Getting the code

tl;dr — give me the code already.

Edit (2011-02-18): the µTP code is now in the Transmission trunk, and the instructions below are no longer current.

Install git (git-core on Debian and Ubuntu) if you don't have it already, and then do

Code: Select all

git clone -b utp git://github.com/jech/transmission.git
cd transmission/
./autogen.sh
make
In the Peers window, peers connected over µTP appear with a T flag (just like on this pretty picture).

1. Low-Priority traffic

BitTorrent is a rather greedy protocol: we're all familiar with how a BitTorrent peer will happily gobble all the bandwidth that's available, leaving none for web browsing or online gaming. The standard solution is to artificially limit the throughput of a BitTorrent peer, typically to 80% or so of your line's available throughput. Doing this is not ideal: on the one hand, we're using just 80% of the available throughput, wasting the rest; on the other hand, the throughput used by BitTorrent remains constant, whle we'd like it to automatically drop when somebody starts browsing the web or playing online games.

The proper solution is to send the BitTorrent traffic at lower priority than the regular traffic, so that it automatically yields to normal priority traffic. Such low-priority, background traffic is sometimes called less than best-effort (LTBE) traffic, since it has lower priority than the normal "best-effort" traffic.

2. µTP

The Micro Transport Protocol (µTP) is BitTorrent, Inc.'s LTBE (low-priority traffic) solution. µTP consists of two parts: a low-priority, low-delay congestion controller called LEDBAT (due to Stanislav Shalunov), and a framing scheme called µTP.

LEDBAT A congestion controller is the piece of code that decides how fast to send packets without overwhelming the network. LEDBAT, the congestion controller used by µTP, is a carefully designed to not interfere with normal "best-effort" traffic, such as gaming and web browsing.

µTP LEDBAT-controlled BitTorrent data is not sent over TCP, as normal BitTorrent data is; instead, the engineers at BitTorrent, Inc. have chosen to put it in UDP packets in a framing that is almost, but not quite, completely unlike TCP. Some people, including myself, have taken a somewhat dim view of the µTP framing.

3. µTP in Transmission

Transmission implements µTP by using the libutp library, written by Greg Hazel and Arvid Norberg, and published as Open Source by BitTorrent, Inc.

The code is currently very preliminary, but usable. Outgoing connexions still need work, as does bandwidth control. And of course we're not going to include our µTP code into Transmission until it has received a lot of testing.

More technical discussion of the µTP code should happen under ticket #2338.

4. Moving LEDBAT forwards

Embedding BitTorrent within µTP allows easy deployment with no modifications to operating system kernels, and serves as a useful proof of concept. In the longer term, however, LTBE is better achieved by putting the LTBE congestion controller in the kernel, to be used with TCP. This has a number of advantages, including:
  • a TCP controller is available to all the programs on the system, not just the BitTorrent implementation; for example, you'd probably want to download OS updates over LTBE;
  • a TCP controller could be used with an unaware peer — you could send LTBE data to a peer that only speaks normal TCP.
As a matter of fact, one LTBE congestion controller is already available in Linux, and I have described elsewhere how to use it with Transmission.

5. Conclusions

LTBE is necessary if we want to do interesting things over the Internet, not just web browsing and e-mail. By investing significant resources into research on LEDBAT, BitTorrent, Inc., have shown that they understand that.

LEDBAT is too important to remain tied to µTP; I'd like to see people push LEDBAT further, and get it to work together with TCP on Windows and Linux.

--jch, Juliusz Chroboczek
jch
Posts: 175
Joined: Wed May 13, 2009 12:08 am

Update: µTP support in the Transmission trunk

Post by jch »

The µTP code has now been merged into the transmission trunk. There is no longer any need to grab it from a third-party repository.

Linux/BSD Unix

The Linux/Unix version is not currently being built with µTP support by default, you'll need to build it yourself. Grab the SVN sources as usual, then do

Code: Select all

./configure --enable-utp
before building.

Mac OS X

The Mac OS X nightlies are being built with µTP support for now (this may change in the future).

Configuration

Go to the Network tab, and enable µTP.

Feedback

As before, feedback of a technical nature should go to https://trac.transmissionbt.com/ticket/2338 .

--jch
Last edited by jch on Fri Feb 18, 2011 3:38 pm, edited 1 time in total.
Jordan
Transmission Developer
Posts: 2312
Joined: Sat May 26, 2007 3:39 pm
Location: Titania's Room

Re: ANNOUNCE: Preliminary µTP support for Transmission

Post by Jordan »

This is a work in progress, so please report any new issues that you find in the nightlies.
gunzip
Posts: 272
Joined: Wed May 05, 2010 2:12 am

Re: ANNOUNCE: Preliminary µTP support for Transmission

Post by gunzip »

jch wrote:Linux/BSD Unix

The Linux/Unix version is not currently being built with µTP support by default, you'll need to build it yourself. Grab the SVN sources as usual, then do

Code: Select all
./configure --enable-utp

before building.
very good jch. building and testing this feature for the first time, i noticed in settings.json a new entry "utp-enabled" was created and was set by default to false. so changing it to:

Code: Select all

"utp-enabled": true
then restarting the daemon, logfile said:

UDP Please add the line "net.core.rmem_max = 4194304" to /etc/sysctl.conf
UDP Please add the line "net.core.wmem_max = 1048576" to /etc/sysctl.conf

did that, updated it with command "sysctl -p" so it would immediately take effect, then restarted the daemon and everything seemed to go fine and no error messages in the log. on first torrent i tried, i saw a lot of "T" flags on the peers, especially with connections to the uTorrent client.

the first couple torrents went very smooth, but is there anything special we should be looking for? and shouldn't both the build and run options be enabled by default to get more people to test it out?
jch
Posts: 175
Joined: Wed May 13, 2009 12:08 am

Re: ANNOUNCE: Preliminary µTP support for Transmission

Post by jch »

gunzip wrote:the first couple torrents went very smooth, but is there anything special we should be looking for?
Except for the upload bandwidth regulator, I consider this code to be production-quality, at least on Linux. So please report anything that looks like a bug.
and shouldn't both the build and run options be enabled by default to get more people to test it out?
Yep. We've been discussing that starting at https://trac.transmissionbt.com/ticket/2338#comment:68 .

--jch
jch
Posts: 175
Joined: Wed May 13, 2009 12:08 am

µTP enabled by default

Post by jch »

µTP is now enabled by default in both the Mac OS X and the generic Unix/Linux binaries. So grab a nightly, and enjoy!

As before, please report any issues on https://trac.transmissionbt.com/ticket/2338 .

--jch
Post Reply