jiigen666/libtorrent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial support for merkle hash tree torrents

Browse Source
This commit is contained in:
Arvid Norberg
2009-03-13 06:09:39 +00:00
parent 8530cb4d88
commit 84a513bffb
15 changed files with 528 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs/features.rst
View File

@@ -46,6 +46,8 @@ extensions
* super seeding/initial seeding (`BEP 16`_).
* private torrents (`BEP 27`_).
* support for IPv6, including `BEP 7`_ and `BEP 24`_.
* support for merkle hash tree torrents. This makes the size of torrent files
scale well with the size of the content.
.. _extensions: manual.html#extensions
.. _`http seeding`: manual.html#http-seeding
@@ -189,6 +191,38 @@ and hence decreasing the likelihood of slow peers blocking the completion of pie
The piece picker can also be set to download pieces in sequential order.
merkle hash tree torrents
-------------------------
Merkle hash tree torrents is an extension that lets a torrent file only contain the
root hash of the hash tree forming the piece hashes. The main benefit of this feature
is that regardless of how many pieces there is in a torrent, the .torrent file will
always be the same size. It will only grow with the number of files (since it still
has to contain the file names).
With regular torrents, clients have to request multiple blocks for pieces, typically
from different peers, before the data can be verified against the piece hash. The
larger the pieces are, the longer it will take to download a complete piece and verify
it. Before the piece is verified, it cannot be shared with the swarm, which means the
larger piece sizes, the slower turnaround data has when it is downloaded by peers.
Since on average the data has to sit around, waiting, in client buffers before it has
been verified and can be uploaded again.
Another problem with large piece sizes is that it is harder for a client to pinpoint
the malicious or buggy peer when a piece fails, and it will take longer to re-download
it and take more tries before the piece succeeds the larger the pieces are.
The piece size in regular torrents is a tradeoff between the size of the .torrent file
itself and the piece size. Often, for files that are 4 GB, the piece size is 2 or 4 MB,
just to avoid making the .torrent file too big.
Merkle torrents solves these problems by removing the tradeoff between .torrent size and
piece size. With merkle torrents, the piece size can be the minimum block size (16 kB),
which lets peers verify every block of data received from peers, immediately. This
gives a minimum turnaround time and completely removes the problem of identifying malicious
peers.
portability
===========

25
docs/make_torrent.rst
View File

@@ -185,7 +185,8 @@ The ``create_torrent`` class has the following synopsis::
struct create_torrent
{
create_torrent(file_storage& fs, int piece_size = 0, int pad_size_limit = -1);
enum { optimize = 1, merkle = 2 };
create_torrent(file_storage& fs, int piece_size = 0, int pad_size_limit = -1, int flags = optimize);
create_torrent(torrent_info const& ti);
entry generate() const;
@@ -211,7 +212,8 @@ create_torrent()
::
create_torrent(file_storage& fs, int piece_size = 0, int pad_size_limit = -1);
enum { optimize = 1, merkle = 2 };
create_torrent(file_storage& fs, int piece_size = 0, int pad_size_limit = -1, int flags = optimize);
create_torrent(torrent_info const& ti);
The ``piece_size`` is the size of each piece in bytes. It must
@@ -220,15 +222,30 @@ piece_size will becalculated such that the torrent file is roughly 40 kB.
If a ``pad_size_limit`` is specified (other than -1), any file larger than
the specified number of bytes will be preceeded by a pad file to align it
with the start od a piece.
with the start od a piece. The pad_file_limit is ignored unless the
``optimize`` flag is passed.
The overlad that takes a ``torrent_info`` object will make a verbatim
The overload that takes a ``torrent_info`` object will make a verbatim
copy of its info dictionary (to preserve the info-hash). The copy of
the info dictionary will be used by ``generate()``. This means
that none of the member functions of create_torrent that affects
the content of the info dictionary (such as ``set_hash()``), will
have any affect.
The ``flags`` arguments specifies options for the torrent creation. It can
be any combination of the following flags:
optimize
This will insert pad files to align the files to piece boundaries, for
optimized disk-I/O.
merkle
This will create a merkle hash tree torrent. A merkle torrent cannot
be opened in clients that don't specifically support merkle torrents.
The benefit is that the resulting torrent file will be much smaller and
not grow with more pieces. When this option is specified, it is
recommended to have a 16 kiB piece size.
generate()
----------