BitTorrentWishList
Jump to navigation
Jump to search
BitTorrent Wish List
These ideas do not express the determined future of BitTorrent. They are ideas made by people in the BitTorrent community who are requesting comments. See related documents below for specifications.
Identification
BitTorrent is a peer-to-peer file sharing protocol designed by Bram Cohen. Visit his pages at [1]. BitTorrent is designed to facilitate file transfers among multiple peers across unreliable networks.
Purpose
This document is intended to be maintained and used by the BitTorrent development community. Everyone is invited to contribute to this document. This is the place to capture detailed descriptions of feature requests, behavorial changes, and other deviations from version 1.0 of the BitTorrent protocol specification.
Related Documents
BitTorrentSpecification - Version 1.0 of the BitTorrent protocol specification.
[2] - The 'official protocol specification.
Contributions
Template Header
I'd like to propose a rough template header, so that we come out of this with some semblance of structure and/or organization. Please use this template header for each proposal that you add.
* Title: Brief (one-line) title for your proposal
* Submitter: Your Yahoo! group handle or name
* Email: Your email address, if you care to share it.
* Category: Category of your proposal. For example, this could be "Feature Request",
"Enhancement", etc. If nothing fits, make something up.
* Compatibilty: Yes/No, yes if your proposal is backwards compatible with the v1.0 spec, no if
it's not. For instance, message additions are backwards compatible if you don't
rely on them for basic functionality in the v2.0 client. Changing the handshake
message format, however, would not be a compatible change.
* Thread: If you've described this request on the BitTorrent group, specify a link to the thread
here. This will provide an easy reference for others to do some background reading.
* Description: Put as much detail in here as you can. Edit what you have as details become
apparent from the discussions on the BitTorrent group.
* Pros: The author and others in the community may add to this section. Basically, this is a
place to summarize the good things about this proposal.
* Cons: The author is likely to not have much to say here, so this section is primarily for the
community. As downsides are identified on the ~Bittorrent group, they should be briefly
summarized here, possibly with references to the threads themselves. This will help to
preserve the final conclusions (if any) for the proposal.
Proposals
Please populate this section.
* Title: Neighbour bitfield, Neighbour HAVE
* Submitter: Vaste
* Email: vaste at vaste mine nu
* Category: Enhancement, Client to client
* Compatibilty: Yes, can be ignored; merely provides better information
* Description: Ability to request/send a histogram summarizing the bitfields of neighbours.
From the perspective of the peer sending the histogram neighbours would be
directly connected peers other than the requesting peer (also excluding self).
The histogram could be encoded in a similar way to a bitfield except the radix of
the histogram would be the maximum value of the histogram + 1, instead of always
2 (binary). (E.g. bitfields 110 + 011 >> histogram 121 >> 16 0x10 encoded,
radix 3; bitfield 101 >> histogram 101 >> 5 0x5 encoded, radix 2.)
Received (normal) HAVEs would be forwarded as Neighbour HAVEs to other peers.
Since change of availability in rare pieces is more important information than in
non-rare pieces, HAVEs of rare-pieces could be forwarded more frequently. Could
(should?) be batched. Should be rationalized, see proposals regarding HAVE
management.
* Pros: Greatly reduces the local blindness of rarity described in Availability, where a
rare piece seems less rare to the peer(s) that can (and likely should) download it. This
should optimize seeding (esp. initial). Gives a slightly better view on availability.
* Cons: Easy to lie. Requires exponential space, time, and bandwidth (highly complex graphs to
correctly model network). Without extreme' care, highly likely to cause broadcast
storms over entire swarm (well known networking problem). Minimal performance gains with
a very* high cost.
* Title: Compact Scrape * Submitter: BORGET S颡stien * Email: sqrtboy@phpsoft.org * Category: Tracker - Bandwidth reduction * Compatibilty: Yes * Thread: http://www.phpsoft.org/trackermodes.php * Description: the main idea is to implement a compact mode for scrape data too. in the thread, I suggest a way to do it. it might need further improvement, but results are already here ! * Pros: saves more bandwith on tracker.
* Cons: need updating bt clients, need more computing power for the bt client
* Title: Get Info / Info * Submitter: Olaf van der Spek * Email* : OvdSpek@LIACS.NL * Category: : Feature Request * Compatibilty: Yes * Thread: http://groups.yahoo.com/group/BitTorrent/message/3889 * Description: Add a pair of messages to allow the exchange of the info section of the metainfo file. With these messages, the metainfo file may substitute an infohash key (with the SHA1 hash of the info key) instead of the info key itself, thus reducing the size of the metainfo file. Also, URIs could be formed which contain the SHA1 infohash, which can be used to request peer lists from the tracker and initiate the getinfo message. See http://sourceforge.net/tracker /?funcdetail&aid842089&groupid84122&atid575157 for details. * Pros: This will relieve much of the bandwidth load on trackers, by allowing the peer network to exchange the info section of the metainfo file. * Cons: This increases the percieved vulnerability of the p2p network to malicious attacks, in the form of erroneous data being introduced to peers requesting the info data from other peers. Olaf: Because the peer data can be verified with the info_hash you get from the tracker, such attacks will fail. * Note: this is implemented as theut_metainfoextension (BEP 9).
* Title: Get Peers / Peers * Submitter: EnergizerRabbit1 * Email: dpmott@sep.com * Category: Feature Request * Compatibilty: Yes * Thread: http://groups.yahoo.com/group/BitTorrent/message/3987 * Description: Add a pair of messages to allow the exchange of peer lists between peers. With these messages, peers may learn of other peers, even if the tracker goes down or disavows the torrent. * Pros: This will relieve some of the bandwidth load on trackers, by allowing the peer network to exchange peer lists. Also, clients could enter the peer network with only one other peer IP address. * Cons: This will have a serious effect on the way the network is connected. Now, the peers you connect to are (uniformly) randomly selected, resulting in a network without cliques of highly connected peers in which some peers are acting as gateways, or bottlenecks, to other cliques. With this feature such cliques and bottlenecks could be created, resulting in e.g. rare pieces being distributed slowly to/from some parts of the network. (A distributed random peer selection algorithm could be used as a substitute, but that is generally expensive and orders of magnitude more complicated.)
It has also been suggested that this feature could be used to initiate DoS attacks against a specific target, and although the likelihood that simultaneous or sustained attacks seems nil, the possibility cannot be ignored. * Note: this is already implemented, at least twice (ut_pex and Azureux PEX).
* Title: Piece compression
* Submitter:
* Email:
* Category: Feature Request
* Compatibilty: Yes, by using options bitfield and not compressing for 1.0 peers
* Thread:
* Description: Provide compression for pieces that are sent on the wire.
* Pros: Peers actually get more than you send, after decompression, effectively increasing you
upload bandwidth.
* Pros: Seeding from offline media like a CDROM ISO would benefit from this. Compression using
different OS and archive application is highly unlikely to yield the exact binary image
needed for seeding, while an ISO rip will.
* Cons: Peers need to be able to tell your client that they can support decompression. Also,
compression takes processing power, which may reduce your overall upload throughput.
* Cons: Highly unlikely to yield useful results. Files being transfered are quite likely to already
be highly compressed with payload-specific compression. The general schemes BT would
use cannot* provide useful additional compression (basic fact of compression methods).
* Re:Cons: Send the piece compressed only if compression saved space, otherwise send uncompressed,
Emule does this. Not sure about CPU/resouce usage.
* Pros: as an *optional* extension, this can actually be pretty useful. BitTorrent is often used
for transferring audio CD images, and those can compress quite well if you use the right
predictor. Most BitTorrent clients do not use much CPU (disk is usually the limiting factor),
and at any rate compression can be disabled when CPU usage is high. The main difficulty is
defining the right set of compression algorithms/predictors, and detecting incompressible data
without actually attempting to compress it.
* Title: Better HAVE message management
* Submitter:
* Email:
* Category: Client Feature Implementation
* Compatibilty: Yes (no protocol change what-so-ever)
* Thread:
* Description: Don't send HAVE messages to peers that already have the piece. (dubbed HAVE
Suppression http://lists.ibiblio.org/pipermail/bittorrent/2005-June/001513.html )
* Pros: Saves your client's upload bandwidth, saves everyone elses download bandwidth. Guarenteed
to provide 50% reduction in number of HAVE messages, roughly 25-33% decrease in
BitTorrent overhead.
* Cons: Weakens the ability of peers to model peer behavior.
* Note: The protocol doesn't strictly require sending HAVE messages when pieces are obtained.
Unless a peer is prepared to verify the peer has the piece by downloading it, you cannot be
certain that the peer has in fact obtained the piece. As such it is worthless to rely on
this information being accurate (smart peers will save bandwdith, malicious peers will lie).
* Title: Get Bitfield / Bitfield, HAVE message management * Submitter: EnergizerRabbit1 * Email: dpmott@sep.com * Category: Feature Request * Compatibilty: No. Sending request is okay (can be ignored by peers), but depending on the response for proper operation is not backwards compatible. * Thread : * Description: Add a pair of messages to allow the exchange of the bitfield which describes available pieces. This will also involve modification to the HAVE message management, notably there needs to be a way to tell the peer to not send the HAVE messages. * Pros: Allows asking for a list of pieces that are available from a peer, instead of receiving HAVE messages. * Cons: Not backwards compatible. Real improvement?
* Title: MULTIHAVE message
* Submitter: Bob
* Email:
* Category: Feature Request
* Compatibilty: Yes, via the options bitfield.
* Thread:
* Description: Instead of sending a HAVE message per piece to everyone, send a MULTIHAVE message
periodically. If too many pieces have been downloaded, then send the bitfield
instead. This feature could also benefit from a way to tell a peer to not send
HAVE, MULTIHAVE, or Bitfield messages.
* Pros: Reduces bandwidth
* Cons: With a well behaving client and reasonable piece size back-to-back HAVE messages will be
uncommon, resulting in trivial bandwidth reduction.
* Note: Most likely implementation, flag in options field. Use length on HAVE message to indicate
advertisement of multiple pieces.
* Title: Done message * Submitter: EnergizerRabbit1 * Email: dpmott@sep.com * Category: Feature Request * Compatibilty: Yes, can always be sent * Thread: * Description: Send a DONE message when download completes. This will allow peers to update their piece bitfields that represent what you have (i.e. everything), and they can start downloading pieces from you. * Pros: Coupled with other 2.0 features, such as the ability to choke HAVE messages from peers, this is a good way to save bandwidth and still get the information you need. Also, this particular enhancement is backwards compatible. * Cons: Useless. Increases protocol complexity. At time of completion you'll of already of advertised almost all messages, therefore advertising multiple pieces at once will be of minimal value. Smart peers can already supress sending HAVE messages for pieces you already have.
* Title: have_ctl (1|0) message * Submitter : EnergizerRabbit1 * Email* : dpmott@sep.com * Category: Feature Request * Compatibilty : Yes, can always be sent * Thread: * Description: Provide a message to enable/disable HAVE messages from peers. * Pros: Allows a peer to manage the (otherwise) unsolicited HAVE messages from peers. * Cons: How can this possibly enhance performance?
* Title: Restructure metadata info file
* Submitter: EnergizerRabbit1
* Email: dpmott@sep.com
* Category: Feature Request
* Compatibilty: No. The original metadata would need to be preserved for 1.0 clients.
* Thread:
* Description: Restructure the metadata file to make it possible to add files to it, without
actually having the other media there. In this way, you could take a dozen or so
individual metadata files, and make a batch torrent out of them, without having the
media on your hard drive. This would involve reorganizing the metadata info to be
file-centric, instead of piece-centric, i.e. each file would have a list of SHA1
hashes, one for each piece.
* Pros: allows creation of batch torrents (i.e. bundles) w/o the original media
* Cons: not backwards compatible with the 1.0 metainfo spec. Also, it can become inefficient if
many of the files are smaller than the chunk size.
* Title: Brief (one-line) No peer ID extension
* Submitter: Olaf van der Spek
* Email: OvdSpek@LIACS.NL
* Category: Tracker - Bandwidth reduction
* Compatibilty: Yes
* Thread: -
* Description: Each client adds &nopeerid=1& to the request URI it sends to the tracker. The
tracker will exclude peer IDs from the output it sends to the client.
* Pros: Tracker bandwidth usage is reduced
* Cons: -
Olaf: The compact tracker protocol already performs this bandwidth reduction, and more. (add
compact=1 to the tracker query URI.)
* Title: Multi Hash Info
* Submitter: Mirco Romanato
* Email: painlord2k@yahoo.it
* Category: Feature - Request
* Compatibilty: Yes
* Thread: -
* Description: Over the usual metadata, insert in the .torrent file the metadata about the single
files (SHA-1, TigerTree, CMD4, MD5). Would be better if this will be
accomplished using magnet links, that can be extended, if needed, without changing
the standard.
* Pros: Sources of data can be located out from the torrent if the application is enabled
(Gnutella/eDonkey200/etc.).
The program could detect if files with the same hashes are present in shared/checked
folders.
This enable to build "virtual" torrent, where there is not real seeder (one or more of the
leechers have only a part of the torrent or the original data will come from outside of the
torrent), but leechers will cluster for the same data, will give priority to the torrent
data exchange and will be able to build a strong request for the missing data with the
external sources (they will ask for the same data in the same time, so there will be a
major chance that the data will leack from the sources and will be served to the other
leechers) . This is like the "horde" system in Overnet.
Trackers are needed only as entry points of the torrent, this will reduce greatly the BW
needing.
* Cons: The last features is dungerous, because could produce a DDoS versus the data sources, if
the applications behaviour is wrong.
* Title: listen_port-message
* Submitter: Arvid Norberg
* Email: c99ang@cs.umu.se
* Category: Feature request
* Compatibilty: Yes
* Thread: -
* Description: When a client connects to another peer and has a green light (a working listen-port)
it tells that peer which port it is listening on. If the client doesn't have a grren
light, or if it just get's an incoming connection, it will not send this information.
* Pros: When a client is stopped, it can save all peers to which it can reconnect to a fast-resume
data file. Without this extension, only 'local' connections can be saved. With this
extension, some 'remote' connections can be saved too. The point of saving the peers ips
and listen-ports is to be able to resume without the need of a tracker, in case it's down.
It will also increase the number of peers you can excange in the presence of "Get Peers /
Peers" extension.
* Cons: a small overhead of a few bytes when connections are opened.
* Note: already implemented as part of LTEP (the 'p' entry).
* Title: Connect To More Seeds Before Connecting To Peers
* Submitter: /dev/null
* Email: devnull2@ownmail.net
* Category: Feature Request/Enhancement
* Compatibilty: Yes, it should possibly be backwords compatible with the v1.0 spec
* Thread: Have not joined the mailing list.
* Description: It seems you always connect to more Peers than Seeds when you start downloading from
the BT network, I believe that it would be better if you were connected to more
Seeds at first and slowly connect to the Peers as the client searches the network
for pieces you need and sends out the pieces you have to trade. Although, the
tracker should connect you to more of the Seeds than it does at the moment, it seems
that you always connect to more Peers than you do Seeds, even if there are more
Seeds in the swarm than Peers. I don't think I am making myself very clear, but
someone could probably figure out what I am trying to explain.
* Pros: None detected.
* Cons: Pointless and wrongheaded. To optimize the overall swarm transfer rate, downloading should
be spread across as many hosts as possible. A swarm of clients preferentially
concentrating their load onto seeds would overload the seeds and underutilize other peers.
As long as the peers your client is connected to hold pieces you don't have, it doesn't
matter whether or not they have a complete set; the client does not have to "search the
network" for pieces it doesn't have, since peers continuously share their piece
availability information.
* Title: BitTorrentAsProtocol * Submitter: Doug Ransom * Email: doug.ransom@alumni.uvic.ca * Category: Tracker - * Compatibilty: No * Thread: * Description: address a resource available via bittorrent with a "URI" such as torrent://tracker:port/announce * Pros: replaces ftp with bittorrent when appropriate. In spirit and purpose of adding new URI schemes. * Cons: Requires some standard way to express all the data normally present in a .torrent file in a form short enough to place in a URI. Likely requires significant protocol changes in order to do so. * Note: already implemented as magnet:btih: URIs.
* Title: user-agent message
* Submitter: WhitSlack
* Email: mwhitlock@whitsoftdev.com
* Category: Enhancement
* Compatibilty: Yes, informational message only, could safely be ignored by non-supporting clients.
* Thread: -
* Description: Add a new message to the protocol to allow a client to identify itself after
handshake and bitfield are sent.
* Pros: Remove dependence on the encoding/decoding of the peerid field for determining remote
peer client software and version. User-agent message would remove space constraints on
client id strings; clients could identify themselves more like HTTP clients (e.g.
"Azureus/2.1.0.4" rather than "-AZ2104-..."). This would simplify coding of clients as the
user agent string could be displayed to the user directly rather than having to parse the
peerid and look up a string to display to the user.
* Cons: Slightly increased overhead. May confuse some poorly written 1.0 clients that don't
ignore messages they don't understand.
* Note: already implemented as part of LTEP (the 'v' entry).
* Title: proxy mode and proxy discovery, aka TorrentSquid * Submitter: DavidNicol * Email: wikitheoryorg@davidnicol.com * Category: Extension * Compatibilty: Yes, Extension, could safely be ignored by non-supporting clients. * Thread: - * Description: Provide a standard way to set up a proxying configuration, in which torrents are all downloaded to a shared torrent server which then serves the files more directly to clients on an internal network, who cannot offer uploading. Users behind impenetrable firewalls would be able to access torrent content, provided their network admins have been kind enough to set such a thing up. Not really a protocol extension, could be handled easily enough with some new software, the proxy server would have a torrent client on it and save all torrents somewhere where the inside systems could get the files via HTTP, and clients would have stub clients that inform the proxy which torrent they are interested in, and the stub waits for the proxy to get the torrent if it doesn't have it already, thenpulls it in from the beginning. The protocol extension would involve a protocol for automatic discovery of a torrent proxy on your network if one exists, which could be as simple as declaring a preferred host name. Everyone using helsinki.example.net for their internet service might look to _torrentproxy.helsinki.example.net for instance, at start-up, before trying to connect to provided tracker URLs directly. Of course a client would remember that no proxy was found and not check again until a week or two had passed.
* Pros: # reduces redundant upstream/downstream torrent traffic within an ISP
# zero configuration through reserving a reccomended DNS name
# definition of ProxyClient - TorrentSquid interface allows interoperability
between ProxyClients and TorrentSquids from different vendors, as well as
unchoked download from local TorrentSquid into standard future torrent client which
can then upload as currently, in non-firewalled environments.
* Cons: # allows censorship at service provider level
# advertises usage of client to ISP via DNS lookup (privacy violation)
* Title: Allow clients to act as "bandwidth donators" (please come up with a better name)
* Submitter: poiutrew
* Category: Enhancement
* Compatibilty: Yes
* Description: In order to improve their upload to download ratio users sometimes download files
that have only a few seeders and many downloading peers, without actually wanting
the file. This is a good thing. But it could be automated. What if users could
select one or more trackers for the client to help when it has unused bandwidth. The
client would then contact the tracker and ask for files that are in need of such
help. The client would download pieces of the file and upload it to other peers.
Only when it cannnot use all of its upload bandwith, would it try to download
another piece.
* Pros: # improves download speeds
* Cons:
* Title: A BitTorrent Private Tracker Client "Accounting Error Message" Protocol Alert
* Submitter: Max Power
* Category: Enhancement
* Compatibility: Can be made compatible with existing clients
* Description: Abstract
Private BitTorrent tracker websites have to obsess over what clients they choose to
connect with.
One of the primary reasons those that run private BitTorrent protocol tracker
websites must obsess over what clients they choose to support is this : so many BT
protocol clients don't correctly provide proper upload or download information to the
tracker.
Without proper accounting data from the client application proper upload and download
ratios can be provided to their (paying) users.
A protocol fix to this unpleasant situation is long overdue.
Scope
This protocol modification is not designed to be used with public trackers -- nor should
it ever be used with public trackers, as public trackers are open to anyone to use and
only engage in limited accounting of the activity of each Torrent. Public trackers most
importantly do not store any user specific upload or download accounting data.
http://hireme.geek.nz/bt-protocol-fix.html
* Pros: Acts as a debugging system for BT clients
* Cons: Increases protocol complexity
Part of