Distributing data on cluster (using torrents?)

[Anthony3]

I hope this is a good place to ask this, otherwise please redirect me to the correct forum. I have a large amount of data (~400GB) I need to distribute to all nodes in a cluster (~100 nodes). Any help into how to do this will be appreciated, following here is what Ive tried. I was thinking of doing this using torrents but I'm running into a bunch of issues. These are the steps I tried:

1. I downloaded ctorrent to create the torrent and seed and download it. I had a problem because I didn't have a tracker.

• I found that qbittorrent-nox has an embedded tracker so I downloaded that on one of my nodes and set the tracker up.

• I now created the torrent using the tracker I created and copied it to my nodes.

• When I run the torrent with ctorrent on the node with the actual data on it to seed the data I get:

  Code: Select all

  Seed for others 72 hours
   - 0/0/1 [1/1/1] 0MB,0MB | 0,0K/s | 0,0K E:0,1 Connecting

• When I run on one of the nodes to download the data I get:
  

  Code: Select all

   - 0/0/1 [0/1/0] 0MB,0MB | 0,0K/s | 0,0K E:0,1

• So it seems they aren't connecting to the tracker ok, but I don't know why

• I am probably doing something very wrong, but I can't figure it out. If anyone can help me with what I am doing, or has any way of distributing the data efficiently, even not with torrents, I would be very happy to hear.

Thanks in advance for any help available.