diff options
Diffstat (limited to 'archive')
| -rw-r--r-- | archive/2022-01-04--meeting-minutes | 8 | ||||
| -rw-r--r-- | archive/2022-01-04-proposal-add-leaf-endpoint | 90 | ||||
| -rw-r--r-- | archive/2022-01-04-proposal-domain-hint | 51 | ||||
| -rw-r--r-- | archive/2022-01-04-proposal-get-endpoints | 46 | ||||
| -rw-r--r-- | archive/2022-01-04-proposal-tree-head-endpoints | 118 | ||||
| -rw-r--r-- | archive/2022-01-18--meeting-minutes | 5 | ||||
| -rw-r--r-- | archive/2022-01-18-proposal-author-reader-terminology | 41 | ||||
| -rw-r--r-- | archive/2022-01-18-proposal-log-url | 26 |
8 files changed, 6 insertions, 379 deletions
diff --git a/archive/2022-01-04--meeting-minutes b/archive/2022-01-04--meeting-minutes index 284c3e5..a828953 100644 --- a/archive/2022-01-04--meeting-minutes +++ b/archive/2022-01-04--meeting-minutes @@ -16,13 +16,13 @@ Status round * [rgdd, ln5] remove arbitrary bytes proposal (re-opened) * https://git.sigsum.org/sigsum/tree/doc/proposals/2021-11-remove-arbitrary-bytes.md * [rgdd, ln5] change get-* endpoints that use HTTP post proposal (new) - * https://git.sigsum.org/sigsum/tree/archive/2022-01-04-proposal-get-endpoints + * https://git.sigsum.org/sigsum/tree/doc/proposals/2022-01-get-endpoints * [rgdd, ln5] stricter domain hint verification proposal (new) - * https://git.sigsum.org/sigsum/tree/archive/2022-01-04-proposal-domain-hint + * https://git.sigsum.org/sigsum/tree/doc/proposals/2022-01-domain-hint * [rgdd, ln5] change add-leaf endpoint proposal (new) - * https://git.sigsum.org/sigsum/tree/archive/2022-01-04-proposal-add-leaf-endpoint + * https://git.sigsum.org/sigsum/tree/doc/proposals/2022-01-add-leaf-endpoint * [rgdd, ln5] change tree-head endpoints proposal (new) - * https://git.sigsum.org/sigsum/tree/archive/2022-01-04-proposal-tree-head-endpoints + * https://git.sigsum.org/sigsum/tree/doc/proposals/2022-01-tree-head-endpoints Decisions * Decision: adopt remove arbitrary bytes proposal diff --git a/archive/2022-01-04-proposal-add-leaf-endpoint b/archive/2022-01-04-proposal-add-leaf-endpoint deleted file mode 100644 index 3123e02..0000000 --- a/archive/2022-01-04-proposal-add-leaf-endpoint +++ /dev/null @@ -1,90 +0,0 @@ -Proposal: change add-leaf endpoint - -Background ---- -Right now a log returns HTTP status 200 OK if it will "try" to merge a submitted -leaf into its Merkle tree. A submitter should not assume that logging happened -until they see an inclusion proof that leads up to a (co)signed tree head. - -If a submitted leaf does not show up in the log despite seeing HTTP status 200 -OK, the submitter must resubmit it. When a resubmission is required/expected is -undefined. - -The reason for this "try" behavior is that log operations become much easier, -especially in self-hosted environments that do not rely on managed databases. -In other words, it is OK to just be "pretty sure" that a submitted leaf will be -persisted and sequenced, and "100%" sure after sequencing actually happened. - -Proposal ---- -A log should not return HTTP status 200 OK unless: -1. The submitted leaf has been sequenced as part of a persisted database. -2. The next tree head that the log signs will contain the submitted leaf. - -HTTP status 3XX is returned with, e.g., "Error=leaf has not been sequenced yet" -if it is not guaranteed that the submitted leaf has been sequenced. - -This means that logging should be assumed after seeing HTTP status 200 OK. This -assumption will be confirmed when the submitter obtains the next (co)signed tree -head. Further investigation is required if it turns out that this assumption is -false. - -Notes ---- -An earlier draft of this proposal considered if useful debug information should -be returned, such as "leaf index", "leaf hash", and "estimated time until a -cosigned tree head is available". We decided to not go in this direction to -avoid redundant and unsigned output that may be mis-used and tampered with ("not -consistent with design"). - -(Note that it is easy to determine when the next cosigned tree head will be -available. The to-sign tree head has a timestamp, and it is rotated every 300s. -Then it takes an additional 300s before the to-sign tree head is served with -collected cosignatures.) - -An earlier draft of this proposal also considered to have verifiable output: - * Option 1: An inclusion proof and a signed tree head - * Option 2: An inclusion proof and a cosigned tree head - -This could be a worthwhile direction if the submitter can only obtain the -required data by using the add-leaf endpoint, thus "forcing resubmits until the -desired output is obtained". Credit to Al Cutter who proposed this (very nice) -idea to us a while back. - -It is not appropriate to always return an inclusion proof for a signed tree -head. What we want is for submitters to get inclusion proofs that reference -cosigned tree heads. - -There are drawbacks to replace the above signed tree head with a cosigned tree -head: - * A submitter that submits multiple leaves will likely (have to?) retrieve - the same cosigned tree head multiple times via the add-leaf endpoint. That - overhead adds up. - * A submitter will have to be in a "resubmit phase" for several minutes as - the default, because it takes time before a cosigned tree head becomes - available. - * (The most sensible implementation would likely resubmit periodically, - say, once per minute. A clever implementation would look at the - timestamp of the to-sign endpoint to determine when is the earliest time - that a merged may have happened.) - -Moreover, removing the get-inclusion-proof and get-tree-head-cosigned endpoints -to force usage of add-leaf excludes (or makes for wonky) usage patterns of the -log: - * "I just want to download all cosigned tree heads to archive them" -> add - leaves. - * "I just want to debug/know that the log is committed to have the leaf - logged, and rely on other witnesses" -> still forced to observe the log's - cosignatures. - * "I want an inclusion proof to a particular tree head" -> build the Merkle - tree yourself to construct that proof. The log's API chooses tree heads for - you. - * (Keeping these endpoints in addition to any new add-leaf output would to - some degree defeat the purpose of adding output, which is why it is not - considered an option.) - -In gist, we decided to go with a solution that is somewhere in between what we -did before and what Al Cutter proposed. We defined when a resubmission is (not) -expected. As a result, a self-hosted log may return at least one HTTP 3XX for -each leaf request, and a few seconds later return HTTP status 200 OK for the -same input data. diff --git a/archive/2022-01-04-proposal-domain-hint b/archive/2022-01-04-proposal-domain-hint deleted file mode 100644 index 322d9cc..0000000 --- a/archive/2022-01-04-proposal-domain-hint +++ /dev/null @@ -1,51 +0,0 @@ -Proposal: stricter domain hint requirements - -Background ---- -Right now a log is expected to look up a submitter's public key hash via DNS. A -domain hint, say, example.com, specifies the location of a TXT RR that contains -the appropriate key hash in hex-encoding. "Some domain knows about the key". - -Downsides with this: -1. A log can be instructed to look up arbitrary TXT records -2. No versioning - -As far as we know there are no amplification threats with (1), but ideally it -would only be possible to query TXT RRs that are actually relevant for Sigsum. - -Not having any versioning could potentially become a headache. All other log -endpoints are versioned. There is no good reason to not have versioning here, -unless that would imply something like registering many different things with -IANA as a result. - -Proposal ---- -Require that a domain hint is formatted as: - - _sigsum_v0.* - -Examples of valid domain hints: - - _sigsum_v0.com - _sigsum_v0.example.com - _sigsum_v0.sub.example.com - -Examples of invalid domain hints: - - _sigsum_v0hello.example.com - -This change addresses both (1) and (2), without making DNS configs harder. - -Notes ---- -For v1 we need to consider if something should be registered with IANA. Credit -to Patrik Wallström who pointed us towards documentation about labels with -underscores: - * https://www.rfc-editor.org/rfc/rfc8552.html - * https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#underscored-globally-scoped-dns-node-names - -Note also that the dependency on TXT look-ups means that a "hidden log" via Tor -would need help from a resolver that is also available over Tor (preferably an -onion but at minimum reachable over TCP). This is because TXT records cannot be -resolved over Tor. This proposal allows the used resolver to be restricted to -only resolve _sigsum_*. diff --git a/archive/2022-01-04-proposal-get-endpoints b/archive/2022-01-04-proposal-get-endpoints deleted file mode 100644 index cbe3170..0000000 --- a/archive/2022-01-04-proposal-get-endpoints +++ /dev/null @@ -1,46 +0,0 @@ -Proposal: change get-* endpoints that use HTTP post - -Background ---- -Right now we HTTP POST ASCII key-value pairs on these endpoints: - * get-leaves - * get-inclusion-proof - * get-consistency-proof - -The original reason was to not have an additional parser, say, input-parameters -with percent-encoding as part of the request URL. - -A major problem with this approach is that it will not be possible to benefit -from HTTP caching. Debugging, with "URLs that reference data" also becomes more -messy. You would have to say "I did printf <stuff> | curl ...". - -Proposal ---- -Change these endpoints so that they use HTTP GET. Encode input params in URL: - - <url>/get-leaves/10/20 # get leaves 10,11,...20 - <url>/get-consistency-proof/10/20 # proof from tree size 10 to 20 - <url>/get-inclusion-proof/10/<leaf hash in hex> # proof for tree size 10 - -This notably avoids percent-encoding which is more messy. - -Notes ---- -We considered if it would be a good idea to re-use our ASCII parser for the -portion of the URL that encodes input data. The basic idea would be that -different "end of key" and "end of value" patterns could be used that are better -suited for a URL. - -For example, instead of (=,\n) one could use ([,]) as ("end of key", "end of value"). - * get-leaves/start_size[10]end_size[12] - * get-consistency-proof/old_size[12]new_size[14] - * get-inclusion-proof/tree_size[10]leaf_hash[ab...ef] - -The reasons why we aborted this direction: - * We can not think of any concrete security risk with the shorter '/' proposal. - * There are very few parameters at play here, hard to confuse and quick - feedback loop if you do. For example, "Error=start size must be smaller or - equal to end size". - * We can be sure that the '/' proposal will not introduce any wonky - interoperability issues; picking a ("end of key", "end of value") would - require much more care. diff --git a/archive/2022-01-04-proposal-tree-head-endpoints b/archive/2022-01-04-proposal-tree-head-endpoints deleted file mode 100644 index b2831bf..0000000 --- a/archive/2022-01-04-proposal-tree-head-endpoints +++ /dev/null @@ -1,118 +0,0 @@ -Proposal: change tree-head endpoints - -Background ---- -Right now the get-tree-head-to-sign endpoint returns the signed tree head that -witnesses should cosign. It does not return any cosignatures. One needs to -wait until the to-sign tree head is finalized and served via -get-tree-head-cosigned. We also have a get-tree-head-latest endpoint that is -sort of hanging around for "debug purposes". - -It would be nice if a submitter could find required cosignatures without always -having to wait for five minutes. The log will likely have received a majority -of cosignatures after one minute, but a submitter currently needs to wait the -full duration before getting access via the get-tree-head-cosigned endpoint. - -It would also be nice to consider if the get-tree-head-latest endpoint can be -removed. - -Here is a rough break-down of how we think about the sigsum API's usage via -roles: - * Submitter - * add-leaf, until HTTP status 200 OK which should mean "you have been sequenced". - * [fetching an inclusion proof for a signed tree head to "verify sequencing" - is not a recommended usage pattern, and does not prevent DoS. The only - difference is that the submitter would notice that the log has not - included with regards to the latest tree head sooner than with regards - to the cosigned tree head. In both cases, there is no proof that - submitter got 200 OK without getting sequenced.] - * Distributor - * get-tree-head-cosigned - * get-inclusion-proof - * [wants "enough" cosignatures, sooner rather than later is a soft requirement] - * Monitor - * get-leaves - * get-tree-head-cosigned - * might hit get-{consistency,inclusion}-proof depending on implementation - * [wants as many cosignatures as possible, does not care about ~minutes of waiting] - * Witness - * get-consistency-proof - * get-tree-head-to-sign - * add-cosignature - * [does not / should not care about other cosignatures; just that the - log signed and that the tree head is consistent with prior history as - observed by the witness] - * End-user - * [does not hit any of the log's endpoints] - * "The curious" - * the latest signed tree head, as fast as possible for quick debug - probably. "is the thing I'm doing working". - * the latest cosigned tree head, with as many cosignatures as possible - for archiving - -Keep in mind that the below proposal should not introduce the log's key hash as -output on any API endpoint. We removed this and other redundant output because -that reduces the risk of faulty implementations that operate on untrusted input. - -For example, in the same way that a faulty witness could verify "the wrong -consistency proof" if it just verified the proof against the tree sizes that the -log returned redundantly (as opposed to the tree sizes the witness asked for), a -faulty witness could end-up cosigning a tree head with another log's context -because "they just copied the key hash and used it because it was there". - -Note that we cannot add the key_hash and cosignature fields to the output of -get-tree-head-to-sign. Our ASCII parser is so simple that it does not permit -empty lists. So, we will either need a way to handle empty lists, or tweak our -endpoints so that they still do what we want without having any empty list. - -[Both rgdd and ln5 would like to avoid complicating the ASCII parser.] - -Proposal ---- -1. Remove the get-tree-head-latest endpoint. We no longer have any recommended -usage-pattern for this endpoint and so it should be removed. Our strongest -arguments for removal are "don't use a signed tree head, it is sort of like a -promise", and "it does not even help you prove that the log's HTTP status 200 OK -semantics were faulty". -2. The get-tree-head-to-sign endpoint is kept as is, but renamed. - * Purpose: used by witnesses. -3. Add an endpoint that returns the logs "to-cosign" tree head and all -cosignatures that were collected thus far. If no cosignatures were received -yet, return an error to avoid having an empty list as output. - * Purpose: used by distributors, but could also be used by a witness' - internal monitoring setup ("is my witness working, are the signatures really - showing up?"). -4. Keep an endpoint that serves the "finalized" cosigned tree head. - * Purpose: mainly used by monitors, but could also be used by distributor's - that don't mind the additional waiting or by parties that want to archive - cosigned tree heads. - -This proposal currently does not have a name for the above endpoints. Help -wanted. - -Notes ---- -A witness polls the "get-tree-head-to-sign" endpoint as before. Witnesses are -recommended to poll the log at least once per minute at randomly selected times. - -After a successful add-cosignature request, a witness should not attempt to add -the same cosignature again. A log can refresh their "to-sign tree head" to -instruct witnesses to send their cosignatures again for the same tree size. - -A witness operator may check that their cosignatures appear on the -"get-tree-head-cosigned endpoints". Such checking would likely be part of how -the operator monitors that the witness operates correctly (i.e., it would not be -something that the witness software does itself after a successful -add-cosignature request). - -A submitter ("Signer" in Figure 1) that wants a cosigned tree head that -satisifies a given policy as fast as possible can poll the "dynamic cosigned -tree head endpoint". Keep in mind that polling more than a few times per minute -would not let you obtain cosignatures much faster, see the above recommendation -for how often witnesses should provide their cosignatures. - -A helpful reflection with regards to naming: - * "The log's to-sign STH shows up, it gets filled-up with cosignatures; the - previous cosigned tree head is served on a separate endpoint. Then - "prev=curr, curr=new". I.e., there is a time aspect here that might be - helpful for naming, although previous and current would be bad choices." diff --git a/archive/2022-01-18--meeting-minutes b/archive/2022-01-18--meeting-minutes index 5040581..fafd4a9 100644 --- a/archive/2022-01-18--meeting-minutes +++ b/archive/2022-01-18--meeting-minutes @@ -31,9 +31,9 @@ Status round * [rgdd, ln5] started sketching on what www.sigsum.org might be eventually * https://git.sigsum.org/sigsum/tree/archive/2022-01-18-future-website-sketch * [rgdd] start using the terminology "author" and "reader" proposal - * https://git.sigsum.org/sigsum/tree/archive/2022-01-18-proposal-author-reader-terminology + * https://git.sigsum.org/sigsum/tree/doc/proposals/2022-01-author-reader-terminology * [rgdd] redefine "base URL" as "log URL" proposal - * https://git.sigsum.org/sigsum/tree/archive/2022-01-18-proposal-log-url + * https://git.sigsum.org/sigsum/tree/doc/proposals/2022-01-proposal-log-url * [ln5, rgdd] continued work on logotype Decisions @@ -42,7 +42,6 @@ Decisions * Author seems like a good abstraction definition * Reader seems less good * Decision: redefine "base URL" as "log URL" - * <insert proposed decision here> Next steps * [rgdd] tree-head endpoint implementation, think about replacement for "reader" diff --git a/archive/2022-01-18-proposal-author-reader-terminology b/archive/2022-01-18-proposal-author-reader-terminology deleted file mode 100644 index fb447d2..0000000 --- a/archive/2022-01-18-proposal-author-reader-terminology +++ /dev/null @@ -1,41 +0,0 @@ -Start using the terminology "author" and "reader" proposal - -Background ---- -Figure 1 in doc/design.md refers to - - a) the party producing a signed checksum as "Signer", and - b) the party verifying a signed checksum as "Verifier". - -This is fine in isolation, but less appropriate when looking at it from a -broader Sigsum perspective. For example, a "Signer" may also be a "Submitter". -It seems like we are mixing terminology for roles and concrete actors here. - -The above is also ambiguous. For example, logs and witnesses sign things; -witnesses and monitors verify things. - -Proposal ---- -1) Replace "Signer" with "Author" when we are talking about a concrete party. - -According to Wikipedia's definition (https://en.wikipedia.org/wiki/Author), an -'author is "the person who originated or gave existence to anything" and whose -authorship determines responsibility for what was created'. This seems -appropriate for us. - -The term "author" has been used in academic litterature before us for similar -purposes: - * "In the setting of transparency logging [18] as depicted in Fig. 1, the - author generates events intended for recipients that describe data - processing by the author as it takes place" - * Link to paper: https://link.springer.com/chapter/10.1007/978-3-319-45741-3_7 - -2) Replace "Verifier" with "Reader" when we are talking about a concrete party. - -According to Wikipedia's definition (https://en.wikipedia.org/wiki/Reading), -"[r]eading is the process of taking in the sense or meaning of letters, symbols, -etc., especially by sight or touch". Although the latter is not a perfect -description for us, the first part is quite close and we could argue that we are -in the "etc" category. - -The main idea here is that it should feel intuitive that an author has readers. diff --git a/archive/2022-01-18-proposal-log-url b/archive/2022-01-18-proposal-log-url deleted file mode 100644 index 598fb43..0000000 --- a/archive/2022-01-18-proposal-log-url +++ /dev/null @@ -1,26 +0,0 @@ -Redefine "base URL" as "log URL" proposal - -Background ---- -The current api.md specification requires that a log has a fixed unique "base -URL". It is any valid HTTP(S) URL that can end with "/sigsum/v0/<endpoint>". - -Proposal ---- -Remove the term "base URL" and instead define "log URL". A log URL is a valid -HTTP(S) URL that ends with "/sigsum/v0/". Example of a valid log URL: - - https://example.com:4711/opposum/sigsum/v0/ - -This means that a named sigsum endpoint can be appended to a log's URL. For -example, if the endpoint is "get-tree-head-quickly" the resulting "endpoint URL" -would be: - - https://example.com:4711/opposum/sigsum/v0/get-tree-head-quickly/ - -And with input parameters for "get-leaves": - - https://example.com:4711/opposum/sigsum/v0/get-leaves/42/4711/ - -Note the final slash in all of the above URLs. Should that be enforced (?). - * XXX: Need to check in URL specification(s). Defer for now. |