Final batch of proposal migrations

i2p2www/spec/proposals/108-multipath-tcp-for-streaming.rst

@@ -0,0 +1,30 @@
+===========================
+Multipath TCP for Streaming
+===========================
+.. meta::
+    :author: hottuna
+    :created: 2012-08-26
+    :thread: http://zzz.i2p/topics/1221
+    :lastupdated: 2012-08-26
+    :status: Draft
+
+.. contents::
+
+
+Introduction
+============
+
+Client tunnels are used by the streaming lib in a fairly standard TCP manner.
+
+It would be preferable to allow a multipath TCP-like solution, where client
+tunnels are used based on individual characteristics like:
+
+- Latency
+- Capacity
+- Availability
+
+
+Proposal
+========
+
+TBD

i2p2www/spec/proposals/118-bidirectional-tunnels.rst

@@ -0,0 +1,58 @@
+=====================
+Bidirectional Tunnels
+=====================
+.. meta::
+    :author: orignal
+    :created: 2016-01-07
+    :thread: http://zzz.i2p/topics/2041
+    :lastupdated: 2016-01-07
+    :status: Draft
+
+.. contents::
+
+
+Introduction
+============
+
+i2pd is going to introduce bi-directional tunnels build through other i2pd
+routers only for now. For the network their will appear as regular inbound and
+outbound tunnels.
+
+Goals
+=====
+
+1. Reduce network and CPU usage by reducing number of TunnelBuild messages
+2. Ability to know instantly if a participant has gone away.
+3. More accurate profiling and stats
+4. Use other darknets as intermediate peers
+
+Tunnel modifications
+====================
+
+TunnelBuild
+-----------
+
+Tunnels are built the same way as inbound tunnels. No reply message is required.
+There is special type of participant called "entrance" marked by flag, serving
+as IBGW and OBEP at the same time. Message has the same format as
+VaribaleTunnelBuild but ClearText contains different fields::
+
+    in_tunnel_id
+    out_tunnel_id
+    in_next_tunnel_id
+    out_next_tunnel_id
+    in_next_ident
+    out_next_ident
+    layer_key, iv_key
+
+It will also contain field mentioning what darknet next peer belong to and some
+additional information if it's not I2P.
+
+TunnelTermination
+-----------------
+
+If peer want to go away it creates TunnelTermination messages encrypts with
+layer key and send in "in" direction. If a participant receive such message it
+encrypts it over with it's layer key and send to next peer. Once a messsage
+reaches tunnel owner it's start decrypting peer-by-peer until gets unencrypted
+message. It finds out which peer has gone away and terminate tunnel.

i2p2www/spec/proposals/119-meta-leaseset-for-multihoming.rst

@@ -0,0 +1,64 @@
+=============================
+Meta-LeaseSet for Multihoming
+=============================
+.. meta::
+    :author: zzz
+    :created: 2016-01-09
+    :thread: http://zzz.i2p/topics/2045
+    :lastupdated: 2016-01-11
+    :status: Draft
+
+.. contents::
+
+
+Introduction
+============
+
+Multihoming is a hack and presumably won't work for e.g. facebook.i2p at scale.
+Say we had 100 multihomes each with 16 tunnels, that's 1600 LS publishes every
+10 minutes, or almost 3 per second. The floodfills would get overwhelmed and
+throttles would kick in. And that's before we even mention the lookup traffic.
+
+We need some sort of meta-LS, where the LS lists the 100 real LS hashes. This
+would be long-lived, a lot longer than 10 minutes. So it's a two-stage lookup
+for the LS, but the first stage could be cached for hours.
+
+
+Format
+======
+
+::
+
+  - Destination
+  - Published Time stamp
+  - Expiration
+  - Flags
+  - Properties
+  - Number of entries
+  - Number of revocations
+
+  - Entries. Each entry contains:
+    - Hash
+    - Flags
+    - Expiration
+    - Cost (priority)
+    - Properties
+
+  - Revocations. Each revocation contains:
+    - Hash
+    - Flags
+    - Expiration
+
+  - Signature
+
+Flags and properties are included for maximum flexibility.
+
+
+Comments
+========
+
+This could then be generalized to be a service lookup of any type. The service
+identifier is a SHA256 hash.
+
+For even more massive scalability, we could have multiple levels, i.e. a meta-LS
+could point to other meta-LSes.

i2p2www/spec/proposals/120-encrypted-leaseset.rst

@@ -0,0 +1,68 @@
+==================
+Encrypted LeaseSet
+==================
+.. meta::
+    :author: zzz
+    :created: 2016-01-11
+    :thread: http://zzz.i2p/topics/2047
+    :lastupdated: 2016-01-12
+    :status: Draft
+
+.. contents::
+
+
+Introduction
+============
+
+Current encrypted LS is horrendous and insecure. I can say that, I designed and
+implemented it.
+
+Reasons:
+
+- AES CBC encrypted
+- Single AES key for everybody
+- Lease expirations still exposed
+- Encryption pubkey still exposed
+
+
+Goals
+=====
+
+- Make entire thing opaque
+- Keys for each recipient
+
+
+Strategy
+========
+
+Do like GPG/OpenPGP does. Asymmetrically encrypt a symmetric key for each
+recipient. Data is decrypted with that asymmetric key. See e.g. [RFC-4880-S5.1]_
+IF we can find an algo that's small and fast.
+
+LS2 contents
+------------
+
+- Destination
+- Published timestamp
+- Expiration
+- Flags
+- Length of data
+- Encrypted data
+- Signature
+
+Encrypted data could be prefixed with some enctype specifier, or not.
+
+Trick is finding an asymmetric encryption that's small and fast. ElGamal at 514
+bytes is a little painful here. We can do better.
+
+See e.g. http://security.stackexchange.com/questions/824...
+
+This works for small numbers of recipients (or actually, keys; you can still
+distribute keys to multiple people if you like).
+
+
+References
+==========
+
+.. [RFC-4880-S5.1]
+    https://tools.ietf.org/html/rfc4880#section-5.1

i2p2www/spec/proposals/121-service-lookup.rst

@@ -0,0 +1,64 @@
+==============
+Service Lookup
+==============
+.. meta::
+    :author: zzz
+    :created: 2016-01-13
+    :thread: http://zzz.i2p/topics/2048
+    :lastupdated: 2016-01-13
+    :status: Draft
+
+.. contents::
+
+
+Introduction
+============
+
+This is the full-monty bombastic anything-goes-in-the-netdb proposal. AKA
+anycast. This would be the 4th proposed LS2 subtype.
+
+Say you wanted to advertise your destination as an outproxy, or a GNS node, or a
+Tor gateway, or a Bittorrent DHT or imule or i2phex or Seedless bootstrap, etc.
+You could store this information in the netDB instead of using a separate
+bootstrapping or information layer.
+
+There's nobody in charge so unlike with massive multihoming, you can't have a
+signed authoritative list. So you would just publish your record to a floodfill.
+The floodfill would aggregate these and send them as a response to queries.
+
+
+Example
+=======
+
+Say your service was "GNS". You would send a database store to the floodfill:
+
+- Hash of "GNS"
+- destination
+- publish timestamp
+- expiration (0 for revocation)
+- port
+- signature
+
+When somebody did a lookup, they would get back a list of those records:
+
+- Hash of "GNS"
+- Floodfill's hash
+- Timestamp
+- number of records
+- List of records
+- signature of floodfill
+
+Expirations would be relatively long, hours at least.
+
+
+Considerations
+==============
+
+The downside is that this could turn into the Bittorrent DHT or worse. At a
+minimum, the floodfills would have to severely rate- and capacity-limit the
+stores and queries. We could whitelist approved service names for higher limits.
+We could also ban non-whitelisted services completely.
+
+Of course, even today's netDB is open to abuse. You can store arbitrary data in
+the netDB, as long as it looks like a RI or LS and the signature verifies. But
+this would make it a lot easier.

i2p2www/spec/proposals/122-new-netdb-entries.rst

@@ -0,0 +1,413 @@
+=================
+New netDB Entries
+=================
+.. meta::
+    :author: zzz
+    :created: 2016-01-16
+    :thread: http://zzz.i2p/topics/2051
+    :lastupdated: 2016-01-16
+    :status: Draft
+
+.. contents::
+
+
+Introduction
+============
+
+This is an update and aggregation of the following 4 proposals:
+
+- LS2
+- Encrypted LS2
+- Meta LS2 for massive multihoming
+- Unauthenticated service lookup (anycasting)
+
+These proposals are mostly independent, but for sanity we define and use a
+common format for several of them.
+
+
+Proposal
+========
+
+This proposal defines 5 new DatabaseEntry types and the process for
+storing them to and retrieving them from the network database,
+as well as the method for signing them and verifying those signatures.
+
+
+Justification
+-------------
+
+LS2 adds fields for changing encryption type and for future protocol changes.
+
+Encrypted LS2 fixes several security issues with the existing encrypted LS by
+using asymmetric encryption of the entire set of leases.
+
+Meta LS2 provides flexible, efficient, effective, and large-scale multihoming.
+
+Service Record and Service List provide anycast services such as naming lookup
+and DHT bootstrapping.
+
+
+Existing types:
+0: RI
+1: LS
+
+New types:
+2: LS2
+3: Encrypted LS2
+4: Meta LS2
+5: Service Record
+6: Service List
+
+Lookup/Store process
+--------------------
+
+Types 2-4 may be returned in response to a standard leaseset lookup (type 1).
+Type 5 is never returned in response to a lookup.
+Types 6 is returned in response to a new service lookup type (type 2).
+
+Format
+------
+
+Types 2-5 all have a common format::
+
+  Standard LS2 Header:
+  - Destination (387+ bytes)
+  - Published timestamp (8 bytes)
+  - Expires (4 bytes) (offset from published in ms)
+  - Flags (2 bytes) (see details for each type below)
+
+  Type-Specific Part
+  - as defined below
+
+  Standard LS2 Signature:
+  - Signature (40+ bytes)
+
+Type 6 (Service List) is an aggregation of several Service Records and has a
+different format. See below.
+
+
+New DatabaseEntry types
+=======================
+
+
+LeaseSet 2
+----------
+
+Changes from existing LeaseSet:
+
+- Add published timestamp, expires timestamp, flags, and properties
+- Add encryption type
+- Remove revocation key
+
+Lookup with:
+    Standard LS flag (1)
+Store with:
+    Standard LS2 type (2)
+Typical expiration:
+    10 minutes, as in a regular LS.
+Published by:
+    Destination
+
+Format
+``````
+::
+
+  Standard LS2 Header:
+  - Destination (387+ bytes)
+  - Published timestamp (8 bytes)
+  - Expires (4 bytes) (offset from published in ms)
+  - Flags (2 bytes)
+
+  Standard LS2 Type-Specific Part
+  - Encryption type (2 bytes)
+  - Encryption key (256 bytes or depending on enc type)
+  - Number of leases (1 byte)
+  - Leases (44 bytes each)
+  - Properties (2 bytes if none)
+
+  Standard LS2 Signature:
+  - Signature (40+ bytes)
+
+Flag definition::
+
+  Bit order: 15 14 ... 2 1 0
+  Bit 0: If 0, a standard published leaseset.
+         If 1, an unpublished leaseset. Should not be flooded, published, or
+         sent in response to a query. If this leaseset expires, do not query the
+         netdb for a new one.
+  Bits 1-15: Unused, set to 0 for compatibility with future uses.
+
+Properties is for future use, no current plans.
+
+Notes
+`````
+- Should we reduce the 8-byte expiration in leases to a 2-byte offset from the
+  published timestamp in seconds? Or 4-byte offset in milliseconds?
+
+- If we ever implement revocation, we can do it with an expires field of zero,
+  or zero leases, or both. No need for a separate revocation key.
+
+
+Encrypted LS2
+-------------
+
+Changes from existing encrypted LeaseSet:
+
+- Encrypt the whole thing for security
+- Securely encrypt, not with AES only.
+- Encrypt to each recipient
+
+Lookup with:
+    Standard LS flag (1)
+Store with:
+    Encrypted LS2 type (3)
+Typical expiration:
+    10 minutes, as in a regular LS.
+Published by:
+    Destination
+
+Format
+``````
+::
+
+  Standard LS2 Header:
+  - Destination (387+ bytes)
+  - Published timestamp (8 bytes)
+  - Expires (4 bytes) (offset from published in ms)
+  - Flags (2 bytes)
+
+  Encrypted LS2 Type-Specific Part
+  - Length of encrypted data (2 bytes)
+  - Encrypted data
+    Format TBD and application-specific.
+    When decrypted, the LS2 Type-Specific part
+
+  Standard LS2 Signature:
+  - Signature (40+ bytes)
+
+Flags: for future use
+
+The signature is of everything above, i.e. the encrypted data.
+
+Notes
+`````
+- For multiple clients, encrypted format is probably like GPG/OpenPGP does.
+  Asymmetrically encrypt a symmetric key for each recipient. Data is decrypted
+  with that asymmetric key. See e.g. [RFC-4880-S5.1]_ IF we can find an
+  algorithm that's small and fast.
+
+  - Can we use a shortened version of our current ElGamal, which is 222 bytes
+    in and 514 bytes out? That's a little long for each record.
+
+- For a single client, we could just ElG encrypt the whole leaseset, 514 bytes
+  isn't so bad.
+
+- If we want to specify the encryption format in the clear, we could have an
+  identifier just before the encrypted data, or in the flags.
+
+- A service using encrypted leasesets would publish the encrypted version to the
+  floodfills. However, for efficiency, it would send unencrypted leasesets to
+  clients in the wrapped garlic message, once authenticated (via whitelist, for
+  example).
+
+- Floodfills may limit the max size to a reasonable value to prevent abuse.
+
+
+Meta LS2
+--------
+
+This is used to replace multihoming. Like any leaseset, this is signed by the
+creator. This is an authenticated list of destination hashes.
+
+It contains a number of entries, each pointing to a LS, LS2, or another Meta LS2
+to support massive multihoming.
+
+Lookup with:
+    Standard LS flag (1)
+Store with:
+    Meta LS2 type (4)
+Typical expiration:
+    Hours to days
+Published by:
+    "master" Destination or coordinator
+
+Format
+``````
+::
+
+  Standard LS2 Header:
+  - Destination (387+ bytes)
+  - Published timestamp (8 bytes)
+  - Expires (4 bytes) (offset from published in ms)
+  - Flags (2 bytes)
+
+  Meta LS2 Type-Specific Part
+  - Number of entries (1 byte)
+  - Entries. Each entry contains: (39 bytes)
+    - Hash (32 bytes)
+    - Flags (2 bytes)
+    - Expires (4 bytes) (offset from published in ms)
+    - Cost (priority) (1 byte)
+
+  - Number of revocations (1 byte)
+  - Revocations: Each revocation contains: (32 bytes)
+    - Hash (32 bytes)
+
+  - Properties (2 bytes if empty)
+
+  Standard LS2 Signature:
+  - Signature (40+ bytes)
+
+Flags and properties: for future use
+
+Notes
+`````
+- A distributed service using this would have one or more "masters" with the
+  private key of the service destination. They would (out of band) determine the
+  current list of active destinations and would publish the Meta LS2. For
+  redundancy, multiple masters could multihome (i.e. concurrently publish) the
+  Meta LS2.
+
+- A distributed service could start with a single destination or use old-style
+  multihoming, then transition to a Meta LS2. A standard LS lookup could return
+  any one of a LS, LS2, or Meta LS2.
+
+- When a service uses a Meta LS2, it has no tunnels (leases).
+
+
+Service Record
+--------------
+
+This is an individual record saying that a destination is participating in a
+service. It is sent from the participant to the floodfill. It is not ever sent
+individually by a floodfill, but only as a part of a Service List. The Service
+Record is also used to revoke participation in a service, by setting the
+expiration to zero.
+
+This is not a LS2 but it uses the standard LS2 header and signature format.
+
+Lookup with:
+    n/a, see Service List
+Store with:
+    Service Record type (5)
+Typical expiration:
+    Hours
+Published by:
+    Destination
+
+Format
+``````
+::
+
+  Standard LS2 Header:
+  - Destination (387+ bytes)
+  - Published timestamp (8 bytes)
+  - Expires (4 bytes) (offset from published in ms, all zeros for revocation)
+  - Flags (2 bytes)
+
+  Service Record Type-Specific Part
+  - Port (2 bytes) (0 if unspecified)
+  - Hash of service name (32 bytes)
+
+  Standard LS2 Signature:
+  - Signature (40+ bytes)
+
+Flags: for future use
+
+Notes
+`````
+- If expires is all zeros, the floodfill should revoke the record and no longer
+  include it in the service list.
+
+- Storage: The floodfill may strictly throttle storage of these records and
+  limit the number of records stored per hash and their expiration. A whilelist
+  of hashes may also be used.
+
+
+Service List
+------------
+
+This is nothing like a LS2 and uses a different format.
+
+The service list is created and signed by the floodfill. It is unauthenticated
+in that anybody can join a service by publishing a Service Record to a
+floodfill.
+
+A Service List contains Short Service Records, not full Service Records. These
+contain signatures but only hashes, not full destinations, so they cannot be
+verified without the full destination.
+
+Lookup with:
+    Service List lookup type (2)
+Store with:
+    Service List type (6)
+Typical expiration:
+    Hours, not specified in the list itself, up to local policy
+Published by:
+    Nobody, never sent to floodfill, never flooded.
+
+Format
+``````
+::
+
+  - Hash of the service name (implicit, in the Database Store message)
+  - Hash of the Creator (floodfill) (32 bytes)
+  - Timestamp (8 bytes)
+
+  - Number of Short Service Records (1 byte)
+  - List of Short Service Records:
+    Each Short Service Record contains (90+ bytes)
+    - Dest hash (32 bytes)
+    - Published timestamp (8 bytes)
+    - Expires (4 bytes) (offset from published in ms)
+    - Flags (2 bytes)
+    - Port (2 bytes)
+    - Sig length (2 bytes)
+    - Signature of dest (40+ bytes)
+
+  - Number of Revocation Records (1 byte)
+  - List of Revocation Records:
+    Each Revocation Record contains (86+ bytes)
+    - Dest hash (32 bytes)
+    - Published timestamp (8 bytes)
+    - Flags (2 bytes)
+    - Port (2 bytes)
+    - Sig length (2 bytes)
+    - Signature of dest (40+ bytes)
+
+  - Signature of floodfill (40+ bytes)
+
+To verify signature of the Service List:
+- prepend the hash of the service name
+- remove the hash of the creator
+- Check signature of the modified contents
+
+To verify signature of each Short Service Record:
+- Fetch destination
+- Check signature of (published timestamp + expires + flags + port + Hash of
+  service name)
+
+To verify signature of each Revocation Record:
+- Fetch destination
+- Check signature of (published timestamp + 4 zero bytes + flags + port + Hash
+  of service name)
+
+Notes
+`````
+- We use signature length instead of sigtype so we can support unknown signature
+  types.
+
+- There is no expiration of a service list, recipients may make their own
+  decision based on policy or the expiration of the individual records.
+
+- Service Lists are not flooded, only individual Service Records are. Each
+  floodfill creates, signs, and caches a Service List. The floodfill uses its
+  own policy for cache time and the maximum number of service and revocation
+  records.
+
+
+References
+==========
+
+.. [RFC-4880-S5.1]
+    https://tools.ietf.org/html/rfc4880#section-5.1

i2p2www/spec/proposals/123-obep-to-one-of-many-tunnels.rst

@@ -0,0 +1,42 @@
+====================================
+OBEP Delivery to One-of-Many Tunnels
+====================================
+.. meta::
+    :author: zzz
+    :created: 2016-03-10
+    :thread: http://zzz.i2p/topics/2099
+    :lastupdated: 2016-03-10
+    :status: Draft
+
+.. contents::
+
+
+Introduction
+============
+
+To reduce connection congestion, give the OBEP a list of id/hash pairs (i.e.
+leases) to deliver the message to rather than just one. The OBEP would select
+one of those to deliver to. The OBEP would select, if available, one that it is
+already connected to, or already knows about.
+
+The originator (OBGW) would stick some (all?) of the target leases in the
+delivery instructions instead of picking just one.
+
+This would make the OBEP-IBGW path faster and more reliable, and reduce overall
+network connections.
+
+Proposal
+========
+
+We have one unused delivery type (0x03) and two remaining bits 0 and 1) in the
+flags. Because we've previously discussed multicast at the OBEP (deliver to all
+specified leases), we could plan for that feature as well at the same time.
+
+So the specification proposal is::
+
+  Flag byte:
+    Delivery type 0x03: count byte and multiple id/hash pairs follow
+  Bit 0: 0 to deliver to one of the tunnels; 1 to deliver to all of the tunnels
+  Count byte: 2-255 number of id/hash pairs to follow (36 bytes each)
+  That many id/hash pairs (36 bytes each)
+  rest of delivery instructions unchanged