BitTorrentWishList

From Theory.org Wiki
Jump to: navigation, search

BitTorrent Wish List

These ideas do not express the determined future of BitTorrent. They are ideas made by people in the BitTorrent community who are requesting comments. See related documents below for specifications.

Identification

BitTorrent is a peer-to-peer file sharing protocol designed by Bram Cohen. Visit his pages at [1]. BitTorrent is designed to facilitate file transfers among multiple peers across unreliable networks.

Purpose

This document is intended to be maintained and used by the BitTorrent development community. Everyone is invited to contribute to this document. This is the place to capture detailed descriptions of feature requests, behavorial changes, and other deviations from version 1.0 of the BitTorrent protocol specification.

Related Documents

BitTorrentSpecification - Version 1.0 of the BitTorrent protocol specification.
[2] - The 'official protocol specification.

Contributions

Template Header

I'd like to propose a rough template header, so that we come out of this with some semblance of structure and/or organization. Please use this template header for each proposal that you add.

* Title: Brief (one-line) title for your proposal
* Submitter: Your Yahoo! group handle or name
* Email: Your email address, if you care to share it.
* Category: Category of your proposal.  For example, this could be "Feature Request", 
            "Enhancement", etc.  If nothing fits, make something up.
* Compatibilty: Yes/No, yes if your proposal is backwards compatible with the v1.0 spec, no if 
                it's not.  For instance, message additions are backwards compatible if you don't 
                rely on them for basic functionality in the v2.0 client.  Changing the handshake 
                message format, however, would not be a compatible change.
* Thread: If you've described this request on the BitTorrent group, specify a link to the thread 
          here.  This will provide an easy reference for others to do some background reading.
* Description: Put as much detail in here as you can.  Edit what you have as details become 
               apparent from the discussions on the BitTorrent group.
* Pros: The author and others in the community may add to this section.  Basically, this is a 
        place to summarize the good things about this proposal.
* Cons: The author is likely to not have much to say here, so this section is primarily for the 
        community.  As downsides are identified on the ~Bittorrent group, they should be briefly 
        summarized here, possibly with references to the threads themselves.  This will help to 
        preserve the final conclusions (if any) for the proposal.

Proposals

Please populate this section.


* Title: Neighbour bitfield, Neighbour HAVE
* Submitter: Vaste
* Email: vaste at vaste mine nu
* Category: Enhancement, Client to client
* Compatibilty: Yes, can be ignored; merely provides better information
* Description: Ability to request/send a histogram summarizing the bitfields of neighbours.
               From the perspective of the peer sending the histogram neighbours would be 
               directly connected peers other than the requesting peer (also excluding self).
               The histogram could be encoded in a similar way to a bitfield except the radix of 
               the histogram would be the maximum value of the histogram + 1, instead of always
               2 (binary). (E.g. bitfields 110 + 011 >> histogram 121 >> 16  0x10 encoded, 
               radix 3; bitfield 101 >> histogram 101 >> 5  0x5 encoded, radix 2.)
                 
               Received (normal) HAVEs would be forwarded as Neighbour HAVEs to other peers.  
               Since change of availability in rare pieces is more important information than in
               non-rare pieces, HAVEs of rare-pieces could be forwarded more frequently. Could
               (should?) be batched. Should be rationalized, see proposals regarding HAVE 
               management.
* Pros: Greatly reduces the local blindness of rarity described in Availability, where a 
        rare piece seems less rare to the peer(s) that can (and likely should) download it. This 
        should optimize seeding (esp. initial). Gives a slightly better view on availability.
* Cons: Easy to lie. Requires exponential space, time, and bandwidth (highly complex graphs to 
        correctly model network). Without extreme' care, highly likely to cause broadcast 
        storms over entire swarm (well known networking problem). Minimal performance gains with 
        a very* high cost.

* Title: Compact Scrape
* Submitter: BORGET S颡stien
* Email: sqrtboy@phpsoft.org
* Category: Tracker - Bandwidth reduction
* Compatibilty: Yes
* Thread: http://www.phpsoft.org/trackermodes.php
* Description: the main idea is to implement a compact mode for scrape data too. in the 
               thread, I suggest a way to do it. it might need further improvement, but results 
               are already here !
* Pros: saves more bandwith on tracker.
* Cons: need updating bt clients, need more computing power for the bt client

* Title: Get Info / Info
* Submitter: Olaf van der Spek
* Email* : OvdSpek@LIACS.NL
* Category: : Feature Request
* Compatibilty: Yes
* Thread: http://groups.yahoo.com/group/BitTorrent/message/3889
* Description: Add a pair of messages to allow the exchange of the info section of the metainfo 
               file.  With these messages, the metainfo file may substitute an infohash key 
               (with the SHA1 hash of the info key) instead of the info key itself, thus 
               reducing the size of the metainfo file.  Also, URIs could be formed which contain 
               the SHA1 infohash, which can be used to request peer lists from the tracker and 
               initiate the getinfo message.  See http://sourceforge.net/tracker
               /?funcdetail&aid842089&groupid84122&atid575157 for 
               details.
* Pros: This will relieve much of the bandwidth load on trackers, by allowing the peer network to 
        exchange the info section of the metainfo file.
* Cons: This increases the percieved vulnerability of the p2p network to malicious attacks, in the 
        form of erroneous data being introduced to peers requesting the info data from other peers.
Olaf: Because the peer data can be verified with the info_hash you get from the tracker, such 
      attacks will fail.
* Note: this is implemented as the ut_metainfo extension (BEP 9).

* Title: Get Peers / Peers
* Submitter: EnergizerRabbit1
* Email: dpmott@sep.com
* Category: Feature Request
* Compatibilty: Yes
* Thread: http://groups.yahoo.com/group/BitTorrent/message/3987
* Description: Add a pair of messages to allow the exchange of peer lists between peers.  With 
               these messages, peers may learn of other peers, even if the tracker goes down or 
               disavows the torrent.
* Pros: This will relieve some of the bandwidth load on trackers, by allowing the peer network to 
        exchange peer lists.  Also, clients could enter the peer network with only one other peer 
        IP address.
* Cons: This will have a serious effect on the way the network is connected. Now, the peers you 
        connect to are (uniformly) randomly selected, resulting in a network without cliques of 
        highly connected peers in which some peers are acting as gateways, or bottlenecks, to other 
        cliques.
        With this feature such cliques and bottlenecks could be created, resulting in e.g. rare 
        pieces being distributed slowly to/from some parts of the network. (A distributed random 
        peer selection algorithm could be used as a substitute, but that is generally expensive and 
        orders of magnitude more complicated.)
        It has also been suggested that this feature could be used to initiate DoS attacks 
        against a specific target, and although the likelihood that simultaneous or sustained 
        attacks seems nil, the possibility cannot be ignored.
* Note: this is already implemented, at least twice (ut_pex and Azureux PEX).

* Title: Piece compression
* Submitter:
* Email:
* Category: Feature Request
* Compatibilty: Yes, by using options bitfield and not compressing for 1.0 peers
* Thread:
* Description: Provide compression for pieces that are sent on the wire.
* Pros: Peers actually get more than you send, after decompression, effectively increasing you 
        upload bandwidth.
* Pros: Seeding from offline media like a CDROM ISO would benefit from this.  Compression using 
        different OS and archive application is highly unlikely to yield the exact binary image 
        needed for seeding, while an ISO rip will.
* Cons: Peers need to be able to tell your client that they can support decompression.  Also, 
        compression takes processing power, which may reduce your overall upload throughput.
* Cons: Highly unlikely to yield useful results. Files being transfered are quite likely to already 
        be highly compressed with payload-specific compression. The general schemes BT would 
        use cannot* provide useful additional compression (basic fact of compression methods). 
* Re:Cons: Send the piece compressed only if compression saved space, otherwise send uncompressed, 
           Emule does this. Not sure about CPU/resouce usage.
* Pros: as an *optional* extension, this can actually be pretty useful.  BitTorrent is often used
        for transferring audio CD images, and those can compress quite well if you use the right
        predictor.  Most BitTorrent clients do not use much CPU (disk is usually the limiting factor),
        and at any rate compression can be disabled when CPU usage is high.  The main difficulty is
        defining the right set of compression algorithms/predictors, and detecting incompressible data
        without actually attempting to compress it.

* Title: Better HAVE message management
* Submitter:
* Email:
* Category: Client Feature Implementation
* Compatibilty: Yes (no protocol change what-so-ever)
* Thread:
* Description: Don't send HAVE messages to peers that already have the piece. (dubbed HAVE 
               Suppression http://lists.ibiblio.org/pipermail/bittorrent/2005-June/001513.html )
* Pros: Saves your client's upload bandwidth, saves everyone elses download bandwidth. Guarenteed 
        to provide 50% reduction in number of HAVE messages, roughly 25-33% decrease in 
        BitTorrent overhead.
* Cons: Weakens the ability of peers to model peer behavior.

* Note: The protocol doesn't strictly require sending HAVE messages when pieces are obtained. 
        Unless a peer is prepared to verify the peer has the piece by downloading it, you cannot be 
        certain that the peer has in fact obtained the piece. As such it is worthless to rely on 
        this information being accurate (smart peers will save bandwdith, malicious peers will lie).

* Title: Get Bitfield / Bitfield, HAVE message management
* Submitter: EnergizerRabbit1
* Email: dpmott@sep.com
* Category: Feature Request
* Compatibilty: No.  Sending request is okay (can be ignored by peers), but depending on the 
                response for proper operation is not backwards compatible.
* Thread : 
* Description: Add a pair of messages to allow the exchange of the bitfield which describes 
               available pieces.  This will also involve modification to the HAVE message 
               management, notably there needs to be a way to tell the peer to not send the HAVE 
               messages.
* Pros: Allows asking for a list of pieces that are available from a peer, instead of receiving 
        HAVE messages.
* Cons: Not backwards compatible. Real improvement?

* Title: MULTIHAVE message
* Submitter: Bob
* Email:
* Category: Feature Request
* Compatibilty: Yes, via the options bitfield.
* Thread: 
* Description: Instead of sending a HAVE message per piece to everyone, send a MULTIHAVE message 
               periodically.  If too many pieces have been downloaded, then send the bitfield 
               instead.  This feature could also benefit from a way to tell a peer to not send 
               HAVE, MULTIHAVE, or Bitfield messages.
* Pros: Reduces bandwidth
* Cons: With a well behaving client and reasonable piece size back-to-back HAVE messages will be 
        uncommon, resulting in trivial bandwidth reduction.
* Note: Most likely implementation, flag in options field. Use length on HAVE message to indicate 
        advertisement of multiple pieces.

* Title: Done message
* Submitter: EnergizerRabbit1
* Email: dpmott@sep.com
* Category: Feature Request
* Compatibilty: Yes, can always be sent
* Thread: 
* Description: Send a DONE message when download completes.  This will allow peers to update their 
               piece bitfields that represent what you have (i.e. everything), and they can start 
               downloading pieces from you.
* Pros: Coupled with other 2.0 features, such as the ability to choke HAVE messages from peers, 
        this is a good way to save bandwidth and still get the information you need.  Also, this 
        particular enhancement is backwards compatible.
* Cons: Useless. Increases protocol complexity. At time of completion you'll of already of 
        advertised almost all messages, therefore advertising multiple pieces at once will be of 
        minimal value. Smart peers can already supress sending HAVE messages for pieces you already 
        have.

* Title: have_ctl (1|0) message
* Submitter : EnergizerRabbit1
* Email* : dpmott@sep.com
* Category: Feature Request
* Compatibilty : Yes, can always be sent
* Thread: 
* Description: Provide a message to enable/disable HAVE messages from peers.
* Pros: Allows a peer to manage the (otherwise) unsolicited HAVE messages from peers.
* Cons: How can this possibly enhance performance?

* Title: Restructure metadata info file
* Submitter: EnergizerRabbit1
* Email: dpmott@sep.com
* Category: Feature Request
* Compatibilty: No.  The original metadata would need to be preserved for 1.0 clients.
* Thread: 
* Description: Restructure the metadata file to make it possible to add files to it, without 
               actually having the other media there.  In this way, you could take a dozen or so 
               individual metadata files, and make a batch torrent out of them, without having the 
               media on your hard drive.  This would involve reorganizing the metadata info to be 
               file-centric, instead of piece-centric, i.e. each file would have a list of SHA1 
               hashes, one for each piece.
* Pros: allows creation of batch torrents (i.e. bundles) w/o the original media
* Cons: not backwards compatible with the 1.0 metainfo spec.  Also, it can become inefficient if 
        many of the files are smaller than the chunk size.

* Title: Brief (one-line) No peer ID extension
* Submitter: Olaf van der Spek
* Email: OvdSpek@LIACS.NL
* Category: Tracker - Bandwidth reduction
* Compatibilty: Yes
* Thread: -
* Description: Each client adds &nopeerid=1& to the request URI it sends to the tracker. The 
               tracker will exclude peer IDs from the output it sends to the client.
* Pros: Tracker bandwidth usage is reduced
* Cons: -
Olaf: The compact tracker protocol already performs this bandwidth reduction, and more. (add 
      compact=1 to the tracker query URI.)

* Title: Multi Hash Info
* Submitter: Mirco Romanato
* Email: painlord2k@yahoo.it
* Category: Feature - Request
* Compatibilty: Yes
* Thread: -
* Description: Over the usual metadata, insert in the .torrent file the metadata about the single 
               files (SHA-1, TigerTree, CMD4, MD5). Would be better if this will be 
               accomplished using magnet links, that can be extended, if needed, without changing 
               the standard.
* Pros: Sources of data can be located out from the torrent if the application is enabled 
        (Gnutella/eDonkey200/etc.).
        The program could detect if files with the same hashes are present in shared/checked 
        folders.
        This enable to build "virtual" torrent, where there is not real seeder (one or more of the 
        leechers have only a part of the torrent or the original data will come from outside of the 
        torrent), but leechers will cluster for the same data, will give priority to the torrent 
        data exchange and will be able to build a strong request for the missing data with the 
        external sources (they will ask for the same data in the same time, so there will be a 
        major chance that the data will leack from the sources and will be served to the other 
        leechers) . This is like the "horde" system in Overnet.
        Trackers are needed only as entry points of the torrent, this will reduce greatly the BW 
        needing.
* Cons: The last features is dungerous, because could produce a DDoS versus the data sources, if 
        the applications behaviour is wrong.

* Title: listen_port-message
* Submitter: Arvid Norberg
* Email: c99ang@cs.umu.se
* Category: Feature request
* Compatibilty: Yes
* Thread: -
* Description: When a client connects to another peer and has a green light (a working listen-port) 
               it tells that peer which port it is listening on. If the client doesn't have a grren 
               light, or if it just get's an incoming connection, it will not send this information.
* Pros: When a client is stopped, it can save all peers to which it can reconnect to a fast-resume 
        data file. Without this extension, only 'local' connections can be saved. With this 
        extension, some 'remote' connections can be saved too. The point of saving the peers ips 
        and listen-ports is to be able to resume without the need of a tracker, in case it's down.
        It will also increase the number of peers you can excange in the presence of "Get Peers / 
        Peers" extension.
* Cons: a small overhead of a few bytes when connections are opened.
* Note: already implemented as part of LTEP (the 'p' entry).

* Title: Connect To More Seeds Before Connecting To Peers
* Submitter: /dev/null
* Email: devnull2@ownmail.net
* Category: Feature Request/Enhancement
* Compatibilty: Yes, it should possibly be backwords compatible with the v1.0 spec
* Thread: Have not joined the mailing list.
* Description: It seems you always connect to more Peers than Seeds when you start downloading from 
               the BT network, I believe that it would be better if you were connected to more 
               Seeds at first and slowly connect to the Peers as the client searches the network 
               for pieces you need and sends out the pieces you have to trade.  Although, the 
               tracker should connect you to more of the Seeds than it does at the moment, it seems 
               that you always connect to more Peers than you do Seeds, even if there are more 
               Seeds in the swarm than Peers.  I don't think I am making myself very clear, but 
               someone could probably figure out what I am trying to explain.
* Pros: None detected.
* Cons: Pointless and wrongheaded.  To optimize the overall swarm transfer rate, downloading should 
        be spread across as many hosts as possible.  A swarm of clients preferentially 
        concentrating their load onto seeds would overload the seeds and underutilize other peers.  
        As long as the peers your client is connected to hold pieces you don't have, it doesn't 
        matter whether or not they have a complete set; the client does not have to "search the 
        network" for pieces it doesn't have, since peers continuously share their piece 
        availability information.

* Title: BitTorrentAsProtocol
* Submitter: Doug Ransom
* Email: doug.ransom@alumni.uvic.ca
* Category: Tracker -
* Compatibilty: No
* Thread:
* Description: address a resource available via bittorrent with a "URI" such as 
                torrent://tracker:port/announce
* Pros: replaces ftp with bittorrent when appropriate.  In spirit and purpose of adding new URI 
         schemes.
* Cons: Requires some standard way to express all the data normally present in a .torrent file in a 
        form short enough to place in a URI. Likely requires significant protocol changes in order 
        to do so.
* Note: already implemented as magnet:btih: URIs.

* Title: user-agent message
* Submitter: WhitSlack
* Email: mwhitlock@whitsoftdev.com
* Category: Enhancement
* Compatibilty: Yes, informational message only, could safely be ignored by non-supporting clients.
* Thread: -
* Description: Add a new message to the protocol to allow a client to identify itself after 
               handshake and bitfield are sent.
* Pros: Remove dependence on the encoding/decoding of the peerid field for determining remote 
        peer client software and version. User-agent message would remove space constraints on 
        client id strings; clients could identify themselves more like HTTP clients (e.g. 
        "Azureus/2.1.0.4" rather than "-AZ2104-..."). This would simplify coding of clients as the 
        user agent string could be displayed to the user directly rather than having to parse the 
        peerid and look up a string to display to the user.
* Cons: Slightly increased overhead. May confuse some poorly written 1.0 clients that don't 
        ignore messages they don't understand.
* Note: already implemented as part of LTEP (the 'v' entry).

* Title: proxy mode and proxy discovery, aka TorrentSquid
* Submitter: DavidNicol
* Email: wikitheoryorg@davidnicol.com
* Category: Extension
* Compatibilty: Yes, Extension, could safely be ignored by non-supporting clients.
* Thread: -
* Description: Provide a standard way to set up a proxying configuration, in which torrents are all 
               downloaded to a shared torrent server which then serves the files more directly to 
               clients on an internal network, who cannot offer uploading. Users behind 
               impenetrable firewalls would be able to access torrent content, provided their 
               network admins have been kind enough to set such a thing up. Not really a protocol 
               extension, could be handled easily enough with some new software, the proxy server 
               would have a torrent client on it and save all torrents somewhere where the inside 
               systems could get the files via HTTP, and clients would have stub clients that 
               inform the proxy which torrent they are interested in, and the stub waits for the 
               proxy to get the torrent if it doesn't have it already, thenpulls it in from the 
               beginning.
               The protocol extension would involve a protocol for automatic discovery of a torrent 
               proxy on your network if one exists, which could be as simple as declaring a 
               preferred host name.  Everyone using helsinki.example.net for their internet service 
               might look to _torrentproxy.helsinki.example.net for instance, at start-up, before 
               trying to connect to provided tracker URLs directly.  Of course a client would 
               remember that no proxy was found and not check again until a week or two had passed.
* Pros: # reduces redundant upstream/downstream torrent traffic within an ISP
        # zero configuration through reserving a reccomended DNS name
        # definition of ProxyClient - TorrentSquid interface allows interoperability 
          between ProxyClients and TorrentSquids from different vendors, as well as 
          unchoked download from local TorrentSquid into standard future torrent client which 
          can then upload as currently, in non-firewalled environments.
* Cons: # allows censorship at service provider level
        # advertises usage of client to ISP via DNS lookup (privacy violation)

* Title: Allow clients to act as "bandwidth donators" (please come up with a better name)
* Submitter: poiutrew
* Category: Enhancement
* Compatibilty: Yes
* Description: In order to improve their upload to download ratio users sometimes download files 
               that have only a few seeders and many downloading peers, without actually wanting 
               the file. This is a good thing. But it could be automated. What if users could 
               select one or more trackers for the client to help when it has unused bandwidth. The 
               client would then contact the tracker and ask for files that are in need of such 
               help. The client would download pieces of the file and upload it to other peers. 
               Only when it cannnot use all of its upload bandwith, would it try to download 
               another piece.
* Pros: # improves download speeds
* Cons:

* Title: A  BitTorrent Private Tracker Client "Accounting Error Message" Protocol Alert 
* Submitter: Max Power
* Category: Enhancement
* Compatibility: Can be made compatible with existing clients
* Description: Abstract
               Private BitTorrent tracker websites have to obsess over what clients they choose to
               connect with.
               One of the primary reasons those that run private BitTorrent protocol tracker
               websites must obsess over what clients they choose to support is this : so many BT
               protocol clients don't correctly provide proper upload or download information to the
               tracker.
               Without proper accounting data from the client application proper upload and download
               ratios can be provided to their (paying) users.
               A protocol fix to this unpleasant situation is long overdue.
               Scope
               This protocol modification is not designed to be used with public trackers -- nor should
               it ever be used with public trackers, as public trackers are open to anyone to use and
               only engage in limited accounting of the activity of each Torrent. Public trackers most
               importantly do not store any user specific upload or download accounting data.
               http://hireme.geek.nz/bt-protocol-fix.html 
* Pros: Acts as a debugging system for BT clients
* Cons: Increases protocol complexity



Part of