BitTorrentLTExtVersionProposal

There is some talks going on about the LText scheme in order to add version and priority between the proposed extension of the handshake. This page is only a tools in order to help people collaborate in its design. DO NOT IMPLEMENT IT. THIS IS NOT FINAL

Introduction

• Description of the 'extension protocol for bittorrent' which is called LT-ext in this document.
• All modifications are done in the handshake of LT-ext

Motivation

The motivation of this effort is to:

1. support of multiple version of a given extension
2. support for priority between different extensions providing a similar feature.

As a requirement: to be backward compatible with client implementing LT-ext without this proposal

Basic Description of the current handshake

• The handshake specifies the extensions supported by the peer transmiting the handshake
• The handshake is specified by a bencoded dictionnary
  • e.g. handshake = { 'm' = { {'goodext', 1}, {'topext', 2} }, 'p' = 4242, 'v' = "SuperClient" }
• The 'm' item of this dictionnary is a dictionnary itself which each extension name as a string and a extended message ID as a integer.
  • e.g. if a Peer-A sends m = { {'goodext', 1}, {'topext', 2} } to Peer-B
  • it means Peer-A supports the extensions 'goodext' and 'topext'
  • if 'goodext' is supported by both peers of the connection, Peer-B will send 'goodext' message with the 'extended message ID' equal to 1
  • if 'topext' is supported by both peers of the connection, Peer-B will send 'topext' message with the 'extended message ID' equal to 2

Proposal to support generic attributes per extension

• lets call this one 'generic attribute proposal' in case other proposals are made.
• version and priority are 'just' attribute of extension.
• this scheme make it generic enougth to later add other attribute
• an element, called 'm_attr' is added to the handshake bencoded dictionnary
• 'm_attr' is a bencoded dictionnary which is optional
  • a remote peer not implementing this proposal would in fact be one peer not providing the 'm_attr' element
• the keys of the 'm_attr' are the extension names, the same names as used in the 'm' element
  • 'm_attr' = { 'goodext' = { 'slot-attr' = 1, 'blip-attr' = 'wowvalue' } }
    • this means that the extension 'goodext' got two attribute 'slota-attr' and 'blip-attr'
    • The name of the attribute keys and the format of their values is to be specified on a per attribute basis.
• relation between 'm' and 'm_attr'
  • any extension name present in 'm_attr' MUST be present in 'm'
  • but extension name present in 'm' may not be in 'm_attr'. (in other words a given extension may not have any attribute)

How to put version as generic attributes

• goal: each peer should be able to support multiple versions of a given extension and to establish preference between them.
• the attribute key 'version' is a bencoded list of integer.
• each integer is the number of a version supported by the peer originating this handshake
• The order of the version define the preference of the originating peer.
  • For example: Peer-A sends a 'version' = [9,2,5] means that Peer-A preferes version 9 over version 2 over version 5.
  • Special case: In case of conflict, the peer which receives the connection, have its preferences satisfied
    • if Peer-A initiates the connection and sends a 'version'=[2,3] and Peer-B replies a 'version'=[3,2], the chosen version is 3.
    • Both accept 2 and 3. but Peer-B is the responder and preferes 3 more than 2.
• TODO to details more and provide more examples

How to put priority between distinct extension providing the same feature

• use case:
  • utpex provides a way to exchange peer information directly between peers without interacting with the http tracker
  • azpex (from azureus) provides similar features
  • if the 2 peers of a connection implement both pex extension, which one to choose ?

• So it is implemented by a attribute called 'impl', short for 'implement this feature'
  • it is a bencoded list of 2 elements
  • the first element is a string for this feature (e.g. 'PEX' for any extension implementing a pex feature)
  • the second element is a integer of the priority for this extension.
• among all extensions with the same 'impl' key (e.g. 'PEX), the one with the highest priority number is prefered

• example
  • 'm_attr' = { 'utpex' = {'impl'=['PEX', 2]}, 'azpex' = {'impl'=['PEX', 3]} }
  • it means that the extensions 'utpex' and 'azpex' both support the 'PEX' features
  • that 'utpex' has a priority of 2 and 'azpex' a priority of 3
  • Among all the extension with the same 'impl' key, in this case 'PEX', the one with the higuest priority number is the choosen one. So in this particular example, azpex
  • and the one with the higuest priority number would be the one

• Similartly to the version attribute, if the priority preference of both peers may lead to multiple choises, the one responding the connection will be the one satisfied
  • (loosely based on the 'usual' principle, in network negociation, the server get to choose)
  • In anycase this is quite arbitrary in a p2p context.

Alternate suggestion

This part describes different behaviour to the one above - it addresses versioning of messages, but does not deal with priorities. Note: I have withdrawn this proposal, but leave it here for completeness.

Description

In the LT handshake, a bencoded dictionary is passed as the payload (as described in the specification document).

One key represents the list of extensions supported, m. This proposal suggests that a new optional key be present to indicate version information, m_ver.

m_ver is a dictionary which contains mappings of extension names to the latest supported version. It is invalid to mention an extension here which is not mentioned in the m dictionary.

Versions will be present as values of the m_ver dictionary, and will be integers. They will start from 1 (not zero), and only increment when a new version of the protocol is added. If a version number is higher than another version number, then this means that the higher version number represents a version of the extension which supersedes the other version of the extension.

If a client supports multiple versions of a message extension, it should mention the *highest* supported version only.

It will be implied that when a client announces that it supports version X, then it also supports all preceeding versions of that extension (1 to X-1). The semantics for dealing with the case when the client does not support earlier versions of a given extension is described later.

If an extension is not mentioned as a key in the m_ver dictionary, or there is no m_ver dictionary present, then it should be assumed that the client only supports the initial version (which is possibly the only version implemented) of the extension, which will be 1.

Agreeing on which version to use

• If two clients indicate that they both support both the same version of the message extension - then that version should be used.
• If two clients do not indicate they support the same version of the message extension, then the highest compatible version should be used (if client A supports v3, and client B supports v2, then it is implictly agreed that v2 should be used - not v1 or v3).
• If two clients do not indicate they support the same version of the message extension, and the client which supports the highest version number does not support the latest version supported by the other client, then another handshake message should be sent from the client to tell the other client that it is dropping support for that extension (as documented in the LT specification document).

Example

Client A submits a handshake with the following dictionary (Python style notation):

 'm': {'ut_pex': 1, 'pi_pex': 2, 'az_pex': 3, 'bc_pex': 4},
 'm_ver': {'ut_pex': 2, 'az_pex': 4}

This means the client supports four extension protocols. The latest supported version of ut_pex is version 2, the latest supported version of az_pex is version 4. pi_pex and bc_pex are only supported up to version 1.

Client B submits a handshake with the following dictionary (Python style notation):

 'm': {'ut_pex': 1, 'bc_pex': 2, 'az_pex': 3},
 'm_ver': {'ut_pex': 1, 'bc_pex': 1}

This means the client supports three extension protocols - all up to version 1.

Let's now say that Client A doesn't support any version of az_pex lower than version 3. Here's the resulting behaviour:

• Both clients support bc_pex at version 1 - so both clients will use bc_pex at version 1.
• Client B does not support the pi_pex extension, so neither client will use that extension with each other.
• Client A supports ut_pex at version 2, but client B only supports version 1, so both clients will use version 1.
• Client A supports az_pex at version 4, but client B only supports version 1. By default, client B will assume that it can send az_pex messages to client A in the version 1 format - but because client A does not support this, it will send another message to client B saying that it is disabling support for that extension.

Withdrawal

After discussing this proposal with Arvid Norberg (author of libtorrent), I have withdrawn this proposal. The main reason for supporting versioning of protocol types was to bring this proposal and the Azureus Messaging Protocol closer together, which could then allow current extensions written for the AzMP to be propogated over the LibTorrent extension protocol.

There are two parts to the version information transferred by the AzMP - the first is in the handshake indicating what the greatest version of an extension you support (which my proposal offered). However, for every packet sent over AzMP, it would contain a version number, indicating what version of the message was being used for that given message - my proposal did not deal with that situation.

An alternative way (which doesn't require any official or unofficial extension to LT-ext) is to use separate extension names for each version of the protocol (something like "AZPEX 1", "AZPEX 2"). Although this would result in a larger number of extensions being announced, it would provide a way of determining the most highly supported version (by iterating over the keys) and would provide a way of sending messages for different versions.

Thanks to Arvid for discussing my proposal with me.