From 533f683ef1ae999c2fdc0086cbc3de4e675d1e33 Mon Sep 17 00:00:00 2001 From: Linus Nordberg Date: Tue, 25 May 2021 11:26:32 +0200 Subject: use POST for requests with input data The major argument for moving input data from HTTP headers in GET requests to body of POST's is that we define the protocol ourselves without any dependencies on HTTP and can make it even simpler to parse. --- doc/api.md | 52 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 32 insertions(+), 20 deletions(-) diff --git a/doc/api.md b/doc/api.md index 4f43d2c..a998d70 100644 --- a/doc/api.md +++ b/doc/api.md @@ -11,11 +11,9 @@ This is a work-in-progress document that may be moved or modified. ## Overview The log implements an HTTP(S) API: -- Requests to the log use the HTTP GET method. -- Input data (in requests) and output data (in responses) are - expressed as ASCII-encoded key/value pairs. -- Requests use HTTP entity headers for input data while responses use - the HTTP message body for output data. +- Input data in requests and output data in responses are expressed as + ASCII-encoded key/value pairs. +- Requests with input data use POST to send the data to the log. - Binary data is hex-encoded before being transmitted. The motivation for using a text based key/value format for request and @@ -136,21 +134,17 @@ constraint is that it must be a valid HTTP(S) URL that can have the URL could be `https://log.example.com/2021/st/v0/get-signed-tree-head`. -Input data (in requests) is sent as ASCII key/value pairs as HTTP -entity headers, with their keys prefixed with the string -`stlog-`. Example: For sending `treee_size=4711` as input a client -would send the HTTP header `stlog-tree_size: 4711`. +Input data (in requests) is POST:ed in the HTTP message body as ASCII +key/value pairs. Output data (in replies) is sent in the HTTP message body in the same format as the input data, i.e. as ASCII key/value pairs on the format -`Key: Value`. Example: For sending `tree_size=4711` as output a log -would send an HTTP message body consisting of `stlog-tree_size: 4711`. +`Key=Value` The HTTP status code is 200 OK to indicate success. A different HTTP -status code is used to indicate failure. The log should set the value -value for the key `error` to a human-readable string describing what -went wrong. For example, `error: invalid signature`, `error: rate -limit exceeded`, or `error: unknown leaf hash`. +status code is used to indicate failure, in which case the log should +respond with a human-readable string describing what went wrong using +the key `error`. Example: `error=Invalid signature.`. ### get-tree-head-cosigned Returns the latest cosigned tree head. Used together with @@ -237,7 +231,7 @@ There is exactly one `signature` and one `key_hash` field. The ### get-proof-by-hash ``` -GET /st/v0/get-proof-by-hash +POST /st/v0/get-proof-by-hash ``` Input: @@ -260,9 +254,12 @@ other words, `SHA256(0x00 | tree_leaf)`. proof of zero or more node hashes. The order of node hashes follow from the hash strategy, see RFC 6962. +Example: `echo "leaf_hash=241fd4538d0a35c2d0394e4710ea9e6916854d08f62602fb03b55221dcdac90f +tree_size=4711" | curl --data-binary @- localhost/st/v0/get-proof-by-hash` + ### get-consistency-proof ``` -GET /st/v0/get-consistency-proof +POST /st/v0/get-consistency-proof ``` Input: @@ -283,9 +280,12 @@ Output on success: consistency proof of zero or more node hashes. The order of node hashes follow from the hash strategy, see RFC 6962. +Example: `echo "new_size=4711 +old_size=42" | curl --data-binary @- localhost/st/v0/get-consistency-proof` + ### get-leaves ``` -GET /st/v0/get-leaves +POST /st/v0/get-leaves ``` Input: @@ -309,9 +309,12 @@ match. The log may return fewer leaves than requested. At least one leaf must be returned on HTTP status code 200 OK. +Example: `echo "start_size=42 +end_size=4711" | curl --data-binary @- localhost/st/v0/get-leaves` + ### add-leaf ``` -GET /st/v0/add-leaf +POST /st/v0/add-leaf ``` Input: @@ -349,9 +352,15 @@ inclusion proof is available. An inclusion proof should not be relied upon unless it leads up to a trustworthy signed tree head. Witness cosigning can make a tree head trustworthy. +Example: `echo "shard_hint=1640995200 +checksum=cfa2d8e78bf273ab85d3cef7bde62716261d1e42626d776f9b4e6aae7b6ff953 +signature_over_message=c026687411dea494539516ee0c4e790c24450f1a4440c2eb74df311ca9a7adf2847b99273af78b0bda65dfe9c4f7d23a5d319b596a8881d3bc2964749ae9ece3 +verification_key=c9a674888e905db1761ba3f10f3ad09586dddfe8581964b55787b44f318cbcdf +domain_hint=example.com" | curl --data-binary @- localhost/st/v0/add-leaf` + ### add-cosignature ``` -GET /st/v0/add-cosignature +POST /st/v0/add-cosignature ``` Input: @@ -369,6 +378,9 @@ head. A key-hash, rather than the full verification key, is used to motivate verifiers to locate the appropriate key and make an explicit trust decision. +Example: `echo "signature=d1b15061d0f287847d066630339beaa0915a6bbb77332c3e839a32f66f1831b69c678e8ca63afd24e436525554dbc6daa3b1201cc0c93721de24b778027d41af +key_hash=662ce093682280f8fbea9939abe02fdba1f0dc39594c832b411ddafcffb75b1d" | curl --data-binary @- localhost/st/v0/add-cosignature` + ## Summary of log parameters - **Public key**: an Ed25519 verification key that can be used to verify the log's tree head signatures. -- cgit v1.2.3 From 8822e78af9fb67dc9280de08c2758350a862b8ab Mon Sep 17 00:00:00 2001 From: Linus Nordberg Date: Tue, 25 May 2021 12:14:45 +0200 Subject: replace some of "the log" and other rephrasing --- doc/api.md | 49 ++++++++++++++++++++++++++----------------------- 1 file changed, 26 insertions(+), 23 deletions(-) diff --git a/doc/api.md b/doc/api.md index a998d70..beda293 100644 --- a/doc/api.md +++ b/doc/api.md @@ -9,11 +9,12 @@ It can be found This is a work-in-progress document that may be moved or modified. ## Overview -The log implements an HTTP(S) API: +Logs implement an HTTP(S) API for accepting requests and sending +responses. - Input data in requests and output data in responses are expressed as ASCII-encoded key/value pairs. -- Requests with input data use POST to send the data to the log. +- Requests with input data use HTTP POST to send the data to a log. - Binary data is hex-encoded before being transmitted. The motivation for using a text based key/value format for request and @@ -27,12 +28,12 @@ wire-format in use by the Tor project. ## Primitives ### Cryptography -The log uses the same Merkle tree hash strategy as +Logs use the same Merkle tree hash strategy as [RFC 6962,ยง2](https://tools.ietf.org/html/rfc6962#section-2). The hash functions must be [SHA256](https://csrc.nist.gov/csrc/media/publications/fips/180/4/final/documents/fips180-4-draft-aug2014.pdf). -The log must sign tree heads using -[Ed25519](https://tools.ietf.org/html/rfc8032). The log's witnesses +Logs must sign tree heads using +[Ed25519](https://tools.ietf.org/html/rfc8032). Log witnesses must also sign tree heads using Ed25519. All other parts that are not Merkle tree related also use SHA256 as @@ -73,7 +74,7 @@ you may use it though. The main point of using Trunnel is that it makes a simple format explicit and unambiguous. #### Merkle tree head -Tree heads are signed by the log and its witnesses. It contains a +Tree heads are signed both by a log and its witnesses. It contains a timestamp, a tree size, and a root hash. The timestamp is included so that monitors can ensure _liveliness_. It is the time since the UNIX epoch (January 1, 1970 00:00 UTC) in seconds. The tree size @@ -93,7 +94,7 @@ not cosign a tree head if it is inconsistent with prior history or if the timestamp is backdated or future-dated more than 12 hours. #### Merkle tree leaf -The log supports a single leaf type. It contains a shard hint, a +Logs support a single leaf type. It contains a shard hint, a checksum over whatever the submitter wants to log a checksum for, a signature that the submitter computed over the shard hint and the checksum, and a hash of the submitter's public verification key, that @@ -113,8 +114,8 @@ struct tree_leaf { ``` `message` is composed of the `shard_hint`, chosen by the submitter to -match the shard interval for the log, and the submitter's `checksum` -to be logged. +match the shard interval for the log it's submitting to, and the +submitter's `checksum` to be logged. `signature_over_message` is a signature over `message`, using the submitter's verification key. It must be possible to verify the @@ -142,13 +143,13 @@ format as the input data, i.e. as ASCII key/value pairs on the format `Key=Value` The HTTP status code is 200 OK to indicate success. A different HTTP -status code is used to indicate failure, in which case the log should +status code is used to indicate failure, in which case a log should respond with a human-readable string describing what went wrong using the key `error`. Example: `error=Invalid signature.`. ### get-tree-head-cosigned Returns the latest cosigned tree head. Used together with -`get-proof-by-hash` and `get-consistency-proof` for verifying the log. +`get-proof-by-hash` and `get-consistency-proof` for verifying the tree. ``` GET /st/v0/get-tree-head-cosigned @@ -306,7 +307,7 @@ value in each list refers to the first leaf, the second value in each list refers to the second leaf, etc. The size of each list must match. -The log may return fewer leaves than requested. At least one leaf +A log may return fewer leaves than requested. At least one leaf must be returned on HTTP status code 200 OK. Example: `echo "start_size=42 @@ -340,11 +341,11 @@ match a hash over `verification_key`. The submission may also not be accepted if the second-level domain name exceeded its rate limit. By coupling every add-leaf request to -a second-level domain, it becomes more difficult to spam the log. You +a second-level domain, it becomes more difficult to spam logs. You would need an excessive number of domain names. This becomes costly if free domain names are rejected. -The log does not publish domain-name to key bindings because key +Logs don't publish domain-name to key bindings because key management is more complex than that. Public logging should not be assumed to have happened until an @@ -373,7 +374,7 @@ Input: Output on success: - None -`key_hash` can be used to identify which witness signed the log's tree +`key_hash` can be used to identify which witness signed the tree head. A key-hash, rather than the full verification key, is used to motivate verifiers to locate the appropriate key and make an explicit trust decision. @@ -382,11 +383,13 @@ Example: `echo "signature=d1b15061d0f287847d066630339beaa0915a6bbb77332c3e839a32 key_hash=662ce093682280f8fbea9939abe02fdba1f0dc39594c832b411ddafcffb75b1d" | curl --data-binary @- localhost/st/v0/add-cosignature` ## Summary of log parameters -- **Public key**: an Ed25519 verification key that can be used to - verify the log's tree head signatures. -- **Log identifier**: the hashed public verification key using SHA256. -- **Shard interval**: the time during which the log accepts logging - requests. The shard interval's start and end are inclusive and - expressed as the number of seconds since the UNIX epoch. -- **Base URL**: where the log can be reached over HTTP(S). It is the - prefix before a version-0 specific endpoint. +- **Public key**: The Ed25519 verification key to be used for + verifying tree head signatures. +- **Log identifier**: The public verification key `Public key` hashed + using SHA256. +- **Shard interval start**: The earliest time at which logging + requests are accepted as the number of seconds since the UNIX epoch. +- **Shard interval end**: The latest time at which logging + requests are accepted as the number of seconds since the UNIX epoch. +- **Base URL**: Where the log can be reached over HTTP(S). It is the + prefix to be used to construct a version 0 specific endpoint. -- cgit v1.2.3 From e374db9e70cd329ff46f1a4443c59a8fa118ddd6 Mon Sep 17 00:00:00 2001 From: Linus Nordberg Date: Fri, 28 May 2021 11:44:39 +0200 Subject: use a proper endpoint in example --- doc/api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api.md b/doc/api.md index beda293..92344c5 100644 --- a/doc/api.md +++ b/doc/api.md @@ -133,7 +133,7 @@ Every log has a base URL that identifies it uniquely. The only constraint is that it must be a valid HTTP(S) URL that can have the `/st/v0/` suffix appended. For example, a complete endpoint URL could be -`https://log.example.com/2021/st/v0/get-signed-tree-head`. +`https://log.example.com/2021/st/v0/get-tree-head-cosigned`. Input data (in requests) is POST:ed in the HTTP message body as ASCII key/value pairs. -- cgit v1.2.3 From fe2e20f346e5f8a66c92016d77f32241498b790e Mon Sep 17 00:00:00 2001 From: Linus Nordberg Date: Fri, 28 May 2021 11:44:54 +0200 Subject: clarify what the signature in get-tree-head-* is covering --- doc/api.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/doc/api.md b/doc/api.md index 92344c5..57ad119 100644 --- a/doc/api.md +++ b/doc/api.md @@ -163,8 +163,9 @@ Output on success: seconds since the UNIX epoch. - `tree_size`: `tree_head.tree_size` ASCII-encoded decimal number. - `root_hash`: `tree_head.root_hash` hex-encoded. -- `signature`: hex-encoded Ed25519 signature over `tree_head` - serialzed as described in section `Merkle tree head`. +- `signature`: hex-encoded Ed25519 signature over `timestamp`, + `tree_size` and `root_hash` serialized into a `tree_head` as + described in section `Merkle tree head`. - `key_hash`: a hash of the public verification key (belonging to either the log or to one of its witnesses), which can be used to verify the most recent `signature`. The key is encoded as defined @@ -192,8 +193,9 @@ Output on success: seconds since the UNIX epoch. - `tree_size`: `tree_head.tree_size` ASCII-encoded decimal number. - `root_hash`: `tree_head.root_hash` hex-encoded. -- `signature`: hex-encoded Ed25519 signature over `tree_head` - serialzed as described in section `Merkle tree head`. +- `signature`: hex-encoded Ed25519 signature over `timestamp`, + `tree_size` and `root_hash` serialized into a `tree_head` as + described in section `Merkle tree head`. - `key_hash`: a hash of the log's public verification key, which can be used to verify `signature`. The key is encoded as defined in [RFC 8032, section 5.1.2](https://tools.ietf.org/html/rfc8032#section-5.1.2), @@ -219,8 +221,9 @@ Output on success: seconds since the UNIX epoch. - `tree_size`: `tree_head.tree_size` ASCII-encoded decimal number. - `root_hash`: `tree_head.root_hash` hex-encoded. -- `signature`: hex-encoded Ed25519 signature over `tree_head` - serialzed as described in section `Merkle tree head`. +- `signature`: hex-encoded Ed25519 signature over `timestamp`, + `tree_size` and `root_hash` serialized into a `tree_head` as + described in section `Merkle tree head`. - `key_hash`: a hash of the log's public verification key that can be used to verify `signature`. The key is encoded as defined in [RFC 8032, section 5.1.2](https://tools.ietf.org/html/rfc8032#section-5.1.2), -- cgit v1.2.3