Create a torrent from a folder with symlinks

[joraff]

Hi all,

I have a special need to create a torrent from a folder that contains symlinks, and have those symlinks be preserved and NOT be traversed. Transmission's current behavior is traverse symbolic links, which in my case is resulting in massive torrent size bloat.

Anyone know how this could be done? Before stating the obvious of either zipping the folder or creating a disk image, which will preserve these symlinks, I'm specifically trying to move away from doing just that.

We use BitTorrent internally for software and operating system deployments. In this context, preserving symlinks would:
- Require half the available disk space
- Reduce disk I/O and deployment times - no need to move the files from the container to their final destination.

Thanks!!

[rb07]

Can't be done with bittorrents.

Preserving symbolic or hard links can only be done with rsync, or similar (unison), and tar (maybe rar, not zip as far as I know).

[Richy_T]

There's no reason the torrent spec couldn't be expanded to allow for storage of symlinks but that might cause issues between different systems (though no doubt there are already a few issues like that unless the spec is very tight).

Maybe consider a script which creates a new dir structure where real files are replaced by symlinks and symlinks completely removed. Generate a script to recreate the symlinks, put it in the new structure then torrent the whole thing. Not as elegant as you'd probably like but it should get you there.

[blacke4dawn]

In my opinion there are two good reasons for torrents to not support symlinks, not all file systems support symlinks and path structure differ between certain OS's.

As for your script idea are you asking for it to be included as standard or asking for opinions on how to make one for your specific needs?

[Richy_T]

Different file structures doesn't necessarily mean that torrent shouldn't support it. There's sane ways to handle incompatibilities. That said, I wouldn't hold my breath.

The script thing was just a suggestion to the original poster as to how he might be able to handle the situation. Though I'm thinking that torrents may be inappropriate for his application anyway.

[blacke4dawn]

Different file structures doesn't necessarily mean that torrent shouldn't support it. There's sane ways to handle incompatibilities. That said, I wouldn't hold my breath.

Was primarily thinking of when one uses absolute paths in the symlink and the effective impossibility of a sane conversion between Windows and Linux/Unix. Besides, you can't change the symlinks them selfs since then you would change the hash info of the pieces that the symlinks are part of.

The script thing was just a suggestion to the original poster as to how he might be able to handle the situation.

Thought so, though took me a few read-through to fully get how you envisioned the script to function.

Though I'm thinking that torrents may be inappropriate for his application anyway.

I'll fully agree with you there.

[Richy_T]

Different file structures doesn't necessarily mean that torrent shouldn't support it. There's sane ways to handle incompatibilities. That said, I wouldn't hold my breath.

Was primarily thinking of when one uses absolute paths in the symlink and the effective impossibility of a sane conversion between Windows and Linux/Unix. Besides, you can't change the symlinks them selfs since then you would change the hash info of the pieces that the symlinks are part of.

Well, I was kinda thinking in terms of "symlinks are not supported under windows" and just not creating them or maybe just creating text files with the sym link info in and leaving it up to the user to sort out. But you are right, I hadn't considered the hashing. I'm still not sure it would be insurmountable (very little is) but I think it would likely be stepping reasonably outside of bittorrenting's scope (distributed file sharing, not distributed file metadata sharing).

[blacke4dawn]

Actually, both symbolic and hard links are supported by NTFS and there are tools for handling them, it just seems they don't want to use them in any way, shape or form.

http://en.wikipedia.org/wiki/NTFS_symbolic_link

[Richy_T]

Actually, both symbolic and hard links are supported by NTFS and there are tools for handling them, it just seems they don't want to use them in any way, shape or form.

http://en.wikipedia.org/wiki/NTFS_symbolic_link

However, they don't seem to be above cluttering my Windows 7 user folder with all sorts of weird links that throw up errors if you click on them. But this isn't really a forum for bashing Windows