TLS SNI problem on OS X

[Crank]

Hi guys,

there is a harsh problem on OS X with ssl trackers using a SNI certificate (like cloudflare does). Some private trackers are using cloudflare for ddos protecting and as we know, cloudflare uses a special certificate system (SNI). Transmission can't handle SNI certificates, so every download from such a tracker ends with a "couldn't connect to tracker" response.
I upgraded my OpenSSL version to 1.0.2g, but nothing happens. What can I do if I want use transmission on trackers whith SNI certificates? I'm using Transmission 2.92 and tested on linux and windows with the same torrent, same tracker and same SNI-certificate. On both operating systems (windows and linux) there aren't any problems. It seems that only on OS X transmission can't handle these special certificates.
Transmission is a great client, so please tell me what I can do (or maybe you to fix this issue). If I disable tls and use the simple https-announce, it works on every system without problems.

Thanks in advance.

[TheBang]

I tested on my own tracker which is running HTTPS via SNI, and there were no problems.

There were, however, problems when using a 301 redirect to rewrite all HTTP traffic (including the announce URL) to HTTPS. This caused an error from the tracker "invalid info hash....". I'm pretty sure this happened because of a parsing error of the serialized torrent data, due to the change in the announce URL (one extra character, due to "http" changing to "https"). When I manually modified the torrent file to https for the announce URL (and incrementing the corresponding index by one), thus avoiding the HTTP to HTTPS redirect, it worked fine.

[Crank]

Did you test on OS X (10.11.3) and a cloudflare certificate? Nevertheless, I took a deeper look into the sourcecode of transmission. Transmission uses a very old cURL library: https://trac.transmissionbt.com/browser ... rl.4.dylib cURL supports SNI certificates on version 7.39 and above. The library in the transmission source is to old. I think in most cases Transmission uses the local cURL installation on the OS, except on OS X, where it doesn't work. To fix this issue, a cURL update (libcurl.4.dylib) would be enough I think. And by the way, it's not only me. I know some other guys with the same problems. OS X (10.11.2/10.11.3), transmission and SNI certificates doesn't work properly. I will create a bug report later.

[mike.dld]

... Nevertheless, I took a deeper look into the sourcecode of transmission. Transmission uses a very old cURL library: https://trac.transmissionbt.com/browser ... rl.4.dylib cURL supports SNI certificates on version 7.39 and above. The library in the transmission source is to old. I think in most cases Transmission uses the local cURL installation on the OS, except on OS X, where it doesn't work. To fix this issue, a cURL update (libcurl.4.dylib) would be enough I think ...

T always uses system libcurl on OS X. There is no complete libcurl library on our SVN, just a stub library (taken from Xcode SDK for OS X ~10.6) to aid in resolving symbols during build. And since openssl API we use is stable enough (despite there being multiple forks recently) and didn't change in years, there is no point in updating this stub library and headers.

Note thought, that you updating openssl (BTW, there's no official way of doing so, so how did you update it exactly?) will have no effect on libcurl as the latter doesn't use openssl on 10.11 and 10.10 (and maybe even earlier versions I don't have at hand, since Apple probably moved to CommonCrypto API and Security framework sometime around 10.7 or 10.8):

Code: Select all

$ otool -L /usr/lib/libcurl.4.dylib
/usr/lib/libcurl.4.dylib:
	/usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 8.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 57330.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1230.0.0)
	/System/Library/Frameworks/LDAP.framework/Versions/A/LDAP (compatibility version 1.0.0, current version 2.4.0)
	/System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos (compatibility version 5.0.0, current version 6.0.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1225.0.0)

So, if you really want to update openssl used by T (not directly (which is another deal), but through libcurl) on OS X you'll need to replace system libcurl as well (on your own risk, I'm not advising anything here); or play with DYLD_LIBRARY_PATH; or build your own T linking statically to both openssl and libcurl.

[Crank]

What has cURL to do with OpenSSL? These are two different librarys and they are doing different jobs. I can access cloudflare protected sites in my browser without problems. It seems that only T has problems with it. And this pointed me to libcurl library. Maybe the local access to openssl and libcurl fails and T uses the files from its source? I'm not an expert in C++ coding, but there is a problem on OS X systems definitely (I'm not the only one). If I've some time I will try to replace the old libcurl version and compile T by myself.
If T would really use the local openssl and cURL on OS X, there would be no problem, because both versions are supporting SNI-Certs.

For updating openssl on OS X I used http://brew.sh/. After installing the new openssl I replaced the binary with the new one. cURL version on my system is 7.43, which is new enough.
Maybe there is a way to get some debug information, I searched but couldn't find information for debug mode on OS X. I think the tls handshake is failing, which results into the "cant connect to tracker" warning.

[mike.dld]

You're talking about trackers. T makes requests to trackers using libcurl (not directly using openssl for this on its own). Now, libcurl doesn't use openssl as I demonstrated. The questions is then, why did you try updating openssl and expect to see the difference?

It's not enough to tell that you have version 7.43. It may be any version, but if it doesn't use openssl then there's no point in stating that openssl supports SNI. Could you (in addition to what x190 suggests) execute `otool` the way I did above on both libcurl and Transmission binaries to make sure that they use the libraries you think they are using and not something else?

Code: Select all

$ otool -L /usr/lib/libcurl.4.dylib
...
$ otool -L /Applications/Transmission.app/Contents/MacOS/Transmission
...

Answering your other question, and reiterating what is already stated before, libraries (.dylib files) on SVN are not full libraries. They only export symbols, but don't implement any functionality, so there is no way those files are used on your system because then nothing would've worked at all.

[Crank]

Code: Select all

$ otool -L /usr/lib/libcurl.4.dylib
/usr/lib/libcurl.4.dylib:
	/usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 8.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 57330.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1230.0.0)
	/System/Library/Frameworks/LDAP.framework/Versions/A/LDAP (compatibility version 1.0.0, current version 2.4.0)
	/System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos (compatibility version 5.0.0, current version 6.0.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1225.0.0)

Code: Select all

$ otool -L /Applications/Transmission.app/Contents/MacOS/Transmission
/Applications/Transmission.app/Contents/MacOS/Transmission:
	/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 22.0.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	@loader_path/../Frameworks/Sparkle.framework/Versions/A/Sparkle (compatibility version 1.5.0, current version 1.5.0)
	@executable_path/../Frameworks/Growl.framework/Versions/A/Growl (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
	/usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
	/usr/lib/libcurl.4.dylib (compatibility version 6.0.0, current version 6.1.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 57337.20.43)
	/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 157.0.0)
	/System/Library/Frameworks/Quartz.framework/Versions/A/Quartz (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1256.1.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 1404.32.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1256.14.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 728.6.0)
	/System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)

I will check the debugging with the environment variables now. If you want test the issue by yourself, I can provide you a torrent file if you want.

[Crank]

Hm, not sure how I'm able to set the "TR_CURL_VERBOSE" to get into the debug mode. Sorry. :/

[Crank]

Thanks for help x190. The result is the following:

Code: Select all

*   Trying 104.24.106.127...
* Connected to xyz.com (104.24.106.127) port 443 (#0)
* Unknown SSL protocol error in connection to xyz.com:-9838
* Closing connection 0
CoreAnimation: warning, deleted thread with uncommitted CATransaction; set CA_DEBUG_TRANSACTIONS=1 in environment to log backtraces.
* Hostname xyz.com was found in DNS cache
*   Trying 104.24.106.127...
* Connected to xyz.com (104.24.106.127) port 443 (#1)
* Unknown SSL protocol error in connection to xyz.com:-9838
* Closing connection 1

Then I used the CA_DEBUG_TRANSACTIONS Flag with results into this:

Code: Select all

CoreAnimation: warning, deleted thread with uncommitted CATransaction; created by:
0   QuartzCore                          0x00007fff95d888b6 _ZN2CA11Transaction4pushEv + 318
1   QuartzCore                          0x00007fff95da3372 _ZN2CA11Transaction15ensure_implicitEv + 276
2   QuartzCore                          0x00007fff95d8ec85 _ZN2CA5Layer13thread_flags_EPNS_11TransactionE + 37
3   QuartzCore                          0x00007fff95d8ebd4 _ZN2CA5Layer4markEPNS_11TransactionEjj + 64
4   QuartzCore                          0x00007fff95d90ae3 _ZN2CA5Layer25set_needs_display_in_rectERK6CGRect + 333
5   QuartzCore                          0x00007fff95d90994 -[CALayer setNeedsDisplayInRect:] + 25
6   AppKit                              0x00007fff990b48f5 _NSBackingLayerSetNeedsDisplayInRect + 319
7   AppKit                              0x00007fff990b47b0 -[_NSBackingLayer setNeedsDisplayInRect:] + 61
8   QuartzCore                          0x00007fff95d90975 -[CALayer setNeedsDisplay] + 62
9   AppKit                              0x00007fff990b4763 -[NSView(NSInternal) _setLayerNeedsDisplayInViewRect:] + 606
10  AppKit                              0x00007fff990984ee NSViewSetNeedsDisplayInRect + 945
11  AppKit                              0x00007fff99098136 -[NSView setNeedsDisplayInRect:] + 48
12  AppKit                              0x00007fff99097f4d -[NSView setNeedsDisplay:] + 81
13  Transmission                        0x000000010dc9bbd0 Transmission + 293840
14  Foundation                          0x00007fff948fdc6f __NSThread__start__ + 1351
15  libsystem_pthread.dylib             0x00007fff88073c13 _pthread_body + 131

[mike.dld]

As I presume you have libcurl installed via homebrew as well, you could try rebuilding it to use the new openssl (if you didn't do that yet) and make T use it, and see if that helps:

Code: Select all

$ brew reinstall curl --with-openssl
$ otool -L /usr/local/opt/curl/lib/libcurl.4.dylib
(confirm that it depends on /usr/local/opt/openssl/lib/... libraries)
$ env DYLD_LIBRARY_PATH=/usr/local/opt/curl/lib /Applications/Transmission.app/Contents/MacOS/Transmission

[Crank]

I will try to rebuild curl later, but I don't think this change anything.

Doubt this has anything to do with Transmission --- Firefox doesn't like that IP either, nor does curl or openssl s_client (no certificate is presented). And something's missing from the logs because xyz.com (now xyz.xyz) doesn't resolve to 104.etc and neither one are trackers? Course my curl and openssl is pretty old, so good luck with the 'homebrew'.

Of course xyz.com isn't the real tracker address. For security reasons I changed this. As I wrote before, I can provide you a torrent file if you want (private message) for testing.

[Crank]

As I presume you have libcurl installed via homebrew as well, you could try rebuilding it to use the new openssl (if you didn't do that yet) and make T use it, and see if that helps:

Code: Select all

$ brew reinstall curl --with-openssl
$ otool -L /usr/local/opt/curl/lib/libcurl.4.dylib
(confirm that it depends on /usr/local/opt/openssl/lib/... libraries)
$ env DYLD_LIBRARY_PATH=/usr/local/opt/curl/lib /Applications/Transmission.app/Contents/MacOS/Transmission

That works! Ok, now is the question, how I can fix this issue permanently without starting T in the terminal?
And what's the best way to fix this problem on other OS X systems (the way above)?

[mike.dld]

As far as I could tell looking into libcurl source code, SChannel (Windows) and DarwinSSL (OS X) backends disable the use of TLS SNI extension unless `verifyhost` is enabled in SSL context; all other backends don't care and could perform SNI validation regardless (hence OpenSSL works). T in turn only enables this option if TR_CURL_SSL_VERIFY environment variable is set (which is not the default), probably because there're lots of self-signed and otherwise invalid certificates out there and validating them will give "Could not connect to tracker" errors (what a coincidence).

Since we now have Let's Encrypt and getting a valid certificate is free and not at all hard, switching the default is something to think about.

[mike.dld]

So here's another test for you. Try running T with TR_CURL_SSL_VERIFY set and see if you get errors from other trackers and/or webseeds:

Code: Select all

$ env TR_CURL_SSL_VERIFY=1 /Applications/Transmission.app/Contents/MacOS/Transmission

[Crank]

Yes, works with that command too without any tracker errors. Only a small warning in terminal appears:

Code: Select all

CoreAnimation: warning, deleted thread with uncommitted CATransaction; set CA_DEBUG_TRANSACTIONS=1 in environment to log backtraces.