There is very little writing on how content creators — whether it be authors, software developers, photographers, or amateur musicians or movie makers — can utilize the BitTorrent protocol to distribute their works. With this tutorial, I hope to help people with the technical bits required to set up a web server to share files over BitTorrent; with a special focus on fully utilizing the distributed aspects of BitTorrent and keeping bandwidth costs down.
This tutorial will show you how you can assure long-term availability of your files distributed through BitTorrent with a low-cost seedbox. For this tutorial, I’ll assume the seedbox will be a cheap virtual private server (may be shared with a web server) or possibly even a Raspberry Pi. The seedbox will help ensure customers actually will get your file and ensures a good user experience over time.
This tutorial is aimed at Fedora Linux, but the instructions are directly applicable on CentOS Red Hat Enterprise Linux (RHEL) as well. Most of the concepts and topics covered in this tutorial will be applicable on all Linux distributions and even other operating systems, although the specific tools to actualize them may vary.
Introduction to distribution via BitTorrent
BitTorrent is a protocol where files that people want to download are cut up into small pieces. Anytime someone wants to download a file, they can download pieces from multiple other people and put it back together using a BitTorrent client. For larger file transfers, this is often much faster than downloading directly from one server that almost always impose bandwidth limits on each user to avoid one user throttling the server. The distributed nature of BitTorrent means no one user, called a ‘peer’, will be carrying all the distribution load and can thus be used to lower bandwidth and server costs while users often get much greater total download speeds from multiple peers.
Distributing files through BitTorrent is very easy. Any client can create torrents, and all you need to do to publish them is to either share a torrent file or a Magnet link with the world. The client you created the torrent on will seed the file to anyone who is interested in downloading it. But how do you share files over BitTorrent and ensure that they will remain available for a long time period? You’ll either need a dedicated commitment to never shutting down your laptop where the client resides, or set up a server to seed the torrent from. I’ll tackle the latter approach.
The peer swarm, the information about who has which part of which file, has been tracked by a centralized server called a tracker. When you create your torrent file, you can embed tracker information inside the file to tell everyone where to go to discover who has the file (or which pieces) and is also online.
The more modern approach of relying on the Distributed Hash Table (DHT), is entirely decentralized and requires no central tracking server. Each user of BitTorrent manages a tiny fraction of the overall DHT network and the users themselves manages the role among themselves that the tracker previously handled.
Distributing through BitTorrent means you give up fine-control over your files distribution, but you receive free bandwidth from everyone who wants to download the file in return. For large files, the cost savings in bandwidth and infrastructure can be enormous for a popular file. By not relying on a tracker, your server doesn’t even need to coordinate the download between users and everything is self-managing. All you need to so is to share your files with the BitTorrent community and continue to share it over time to ensure long-time availability of your files. That is where this tutorial comes in.
Note that as the requirements stated above may suggest, this setup will not provide fast transfer speeds but only act as a low-cost and slow seeder that can remain available over time. You can experiment with increasing memory, peer, and bandwidth limits if you want to ensure fast download as well as availability.
Installing and configuring Transmission
No special software is needed to share a file on BitTorrent and any client can be used. I’ve found the Transmission client very to be the best alternative for use on a server as it has good compatibility with other clients, support all the recent protocol standards, is easy to use, and has a low memory footprint.
To get started, install the Transmission daemon (non-graphical variant for server use) from your package manager and start it once to create the default files and directories that you’ll need to work with later.
Then create the /var/lib/transmission/seeding directory and assign it to the new “transmission” user account that was created when you installed Transmission:
You should now have a set of directories and default configuration files for Transmission. Make sure you stop the Transmission service before adjusting Transmission’s main configuration file. Otherwise your changes will be overwritten when Transmission exits.
Then proceed to open up and modify /var/lib/transmission/.config/transmission-daemon/settings.json in your favorite text editor. It should already contain all the default configuration options in their default states. Here is a synopsis of the changes that you should make to the defaults:
The rest of this section will cover each of these options in more depth. You may wish to skip ahead to the next section if you feel comfortable that you understood the above options and values.
watch-dir and watch-dir-enabled: Transmission will look for new torrent files in the destination directory, and will add these to the downloading and seed list. This directory is where you’ll add new torrents later that you wish to distribute.
peer-limit-for-torrent and upload-slots-per-torrent: The first value should be approximately double the value of the latter. The peer limit controls how many people your server is willing to communicate to about the torrent, and the latter how many it is willing to upload it to at any given time. These values should remain low even when interest for the files are high as the distributed nature of BitTorrent means that other people will also be distributing your files on your behalf. Keeping the values low is one of the most important metrics for keeping Transmission’s memory usage low.
pex-enable: With Peer-Exchange (PEX) enabled, both people with and without an upload slot will be informed about other people who have the entire or partial file available. By sharing this information, people can more easily discover other users who do have available upload slots. Asking users who already know about other users who has a file is usually more efficient than the slower to act DHT network. This system is why your peer queue limit should be higher than your upload slot limit.
speed-limit-down and speed-limit-up: Controls the maximum upload and download bandwidth in KiB. Setting this low many reduce your bandwidth costs as clients will be more likely to chose other peers when available over your slower peer. As you’ll not wish to bear the brunt of the distribution bandwidth yourself, these should only be increase if you have more bandwidth and memory to spear.
queue-stalled-enabled: Set to disabled to prevent Transmission from ever stopping to seed your files in periods when there is low interest in your files. As Transmission is serving as a long-term availability distribution node for your files, it shouldn’t really ever pause any of your torrents.
dht-enabled: Register your client and torrents in the global distributed hash table (DHT) network so that peers can locate you as a seeder. On by default, and you shouldn’t turn this off even if you’ve chosen to use a tracker.
More information about Transmission’s many configuration options is available from the ConfigFiles wiki page.
Configuring the system for Transmission
Next we’ll tweak some of the system configurations for Transmission including the system service file (‘initscript’) and the system firewall. We’ll start by adjusting the system service file to keep Transmission’s footprint as minimal as possible.
If you’re not intending to run Transmission on a low-end system or have unlimited bandwidth, you’ll want to carefully review the man pages of systemd.exec and systemd.resource-control. For everyone using anything between a reasonably priced hosted server and a Raspberry Pi, the below suggestions should work great.
Start by copying the system-provided systemd service file for Transmission to a the location for customized service files to avoid having your modifications overwritten with future system updates:
The open up and modify /etc/systemd/system/transmission-daemon.service in your favorite text editors. Add these statements at the bottom of the existing [Service] section of the file (omitting the ellipsis):
These options specify that the process is to run with low system priority and with low impact on the CPU and disk, as well as keep the memory usage within low parameters. This will prevent Transmission from eating all your available system resources.
After installing the new copy of the service file, you’ll need to reload the systemd service cache in order for the new file to be visible to the system:
The last step before we start the Transmission service and start looking into adding torrents, is to allow BitTorrent to talk through the system firewall (FirewallD):
Note that the default transmission-client service only opens up TCP port 51413. The DHT and µTP protocol extensions require UDP to be available on the same port well. Pending a fix for this in firewalld, you must manually include UDP port 51413 as shown below when adding the transmission service to your firewall rules.
If there are any intermediary proxies, firewalls, or NAT devices you may need to configure port-forwarding or otherwise allow TCP/UDP traffic with your server on port 51413.
You’re now ready to start up the transmission-daemon, and if that runs with no errors continue to enable the new service as a default service that will start on every system boot:
By now, you should have gotten yourself a new BitTorrent seedbox server!
Adding torrents to seed
A BitTorrent seedbox is only as good as the torrent it distributes. You can create a torrent form any file using any BitTorrent client such as Transmission (available to all operating systems with graphical user-interface), or using a specialist tool to generate the torrent such as my mitorrent program for generating torrents. You shouldn’t set a tracker address as your seedbox should be discoverable over DHT. Operating a tracker is a very bandwidth and server resource intensive task, and with DHT there is no longer any need for it either.
As we configured Transmission to watch for new torrents and to store its downloaded data in the /var/lib/transmission/seeding directory, you can simply place your torrent files and data files in that directory.
Alternatively, you can start seeding from a BitTorrent client on your own PC transfer the data files and the torrent file itself over BitTorrent. To tell your seedbox about the torrents you want, execute the transmission-remote command and add the torrent’s Magnet link. You can find this by right-clicking on transfer item in most BitTorrent clients and choosing “Copy Magnet link”. This next example command will add and download my Trackerless Magnet URI browser extension to your seedbox to demonstrate how you can add a Magnet link:
If you’ve set up multiple seed boxes, say at different data centers around the world, then you only need to send either the torrent file or Magnet link to each server. They’ll download and distribute the data files among themselves as soon as one of them — or even your local BitTorrent client — starts sharing the file. It’s quite quick and easy to scale. Now you’re thinking with torrents.
You can share your files with your customers either by giving them a traditional torrent file, or by sharing the Magnet URI link. I’d recommend you use torrent files over Magnet links only if you work with legacy or restricted web publishing systems that don’t allow you to publish a Magnet URI. (I’m looking at you WordPress!)
Please also include a small help text such as “A BitTorrent client program is required to download the file” next to the torrent file or Magnet link on your website and other distribution channels. You can help out your users by linking out to and recommending a BitTorrent client as well. Please keep in mind that you should recommend a BitTorrent client that will work with the customer’s devices, so recommend one for each popular operating system.
If you’re not willing or able to configure and host your own files, you may want to consider some hosted BitTorrent distribution options.
Amazon Simple Storage Server (S3) offers zero-configuration BitTorrent that allow you distribute your files via Amazon’s BitTorrent trackers. This is only an option if you want to use stateful tracker-based peer discovery as Amazon doesn’t publish your torrents to the DHT network.
Amazon charge per gigabyte/month of storage, plus per gigabit transfer from their servers. Costs will be reduced by the distributed nature of BitTorrent, but as Amazon control the tracker and are financially motivated to serve your files themselves … let us just say I wouldn’t be surprised if the tracker favors Amazon’s hosted peers that they can charge for over other peers.
Mininova offers the cost-free Mininova Content Distribution for “high quality content producers”. It’s unclear who this is aimed at, but they only seem to accept ebooks, music, and video produces. If you’re interested in distributing other materials through them, I’d suggest you contact them and ask anyway.
Like Amazon S3, Mininova will host and seed your files through their own BitTorrent tracker but not over DHT. This is again a weird decision as I would believe that they’d be interested in keeping costs down as the service is provided free of charge. I contacted Mininova to ask how long they host the content, and was told they’ll host it indefinitely but noted that “[Mininova] might start dropping old content that nobody is downloading”. Amazon or your own seedbox seem to be a more reliable long-term option, though Mininova definitely has the lowest barrier to entry.
I hope this was enough to get you started with distributing your digital material over BitTorrent. Distributed networks are really cool, and especially when they give your customers faster downloads and help save you money at the same time!