diff options
Diffstat (limited to 'archive')
| -rw-r--r-- | archive/2021-11-09-ascii-parser-aborted-idea | 60 | ||||
| -rw-r--r-- | archive/2021-11-09-sharding-ideas | 55 |
2 files changed, 115 insertions, 0 deletions
diff --git a/archive/2021-11-09-ascii-parser-aborted-idea b/archive/2021-11-09-ascii-parser-aborted-idea new file mode 100644 index 0000000..765976c --- /dev/null +++ b/archive/2021-11-09-ascii-parser-aborted-idea @@ -0,0 +1,60 @@ +Background +--- +Is it possible to make our ascii parser simpler and less error-prone for lists? + +Suppose we have a list of items named "apples". Each item contains a single value. +The list apples=[v1,v2] would be expressed as follows: +``` +apples=v1 +...possibly other key-value pairs... +apples=v2 +``` + +Suppose we have a list of items named "fruits". Each fruit item contains two values +for the fields "apple" and "banana". The list fruits=[(apple=v1,banana=v2), (apple=v3,banana=v4)] could be expressed as follows: +``` +banana=v2 +apple=v1 +apple=v3 +banana=v4 +``` + +To parse this successfully, we require that the number of apple lines are the same as +the number of banana lines. Then we join apple-banana pairs in the observed ordered, +i.e., the first apple line is associated with the first banana line, etc. + +Observation +--- +Would it be simpler to parse the following: + +``` +fruits=apple-->v1; banana-->v2; +fruits=banana-->v4; apple-->v3; +``` + +In other words, the list is identified by a key. An item's key-value pairs are listed +on the same line, and uses _the same ASCII parser but with different delimitors_. + +To improve readability for humans, the delimiting pattern could be based on k, k-1, +k-2,...,1 new lines. The same technique could be used for the end-of-key pattern. +``` +key==key1=v1 +key2=v3 + +key==key1=v2 +key2=v4 + +``` + +Discussion +--- +Not convinced that nested parsers are simpler than joining lists after checking that +all sizes match. It might be easier to look at for humans, because we get each item +grouped in the same place. Our parser description would become more complex, however. + +Assuming that the parser's syntax is understood, it might be the case that it is easier +to implement generally in a high-level language. That is a non-goal though. + +Conclusion +--- +This idea does not seem to be simpler and less error-prone in general. Abort! diff --git a/archive/2021-11-09-sharding-ideas b/archive/2021-11-09-sharding-ideas new file mode 100644 index 0000000..535855e --- /dev/null +++ b/archive/2021-11-09-sharding-ideas @@ -0,0 +1,55 @@ +Warning: rough sketch, not considered in full detail yet. + +Background +--- +The process around shard rotation can probably be simplified for all involved parts. + +Right now, a log's metadata is fixed and consists of: + * A unique base URL + * A unique public key + * A shard start + * A shard end + +This means that new log metadata needs to be delivered to all parts somehow. + * Potentially painful for hard-to-update verifiers, like in a firmware context + * Potentially annoying and error-prone for all involved parties to track + +Ideas +--- +1. Introducing new shards is like introducing new log operators. You could announce +many log shards at the same time, or use a dynamic list of log operators and shards. + * Con: announcing many shards at once involves tracking and securing more keys + * Con: a dynamic list is a moving part that can break and/or is hard in some contexts + +2. Let transition from one shard to the next be easily automated in the normal case: + * Remove requirement of unique public key, rework "log id" so it is unique per shard + * Log adds "if_next_shard_url", e.g., in its metadata that is served on an endpoint + * Witnesses periodically check if a new shard was started. If yes: + * Locate the new metadata on <url>/log-metadata + * Log a signed checksum of the new metadata in the current shard + * Witness: "look, next shard is available. I will start cosigning that too." + * Monitor: + * "okay, checking if_next_shard_url and if your metadata claim is valid" + * "great shard is there, will start monitoring that as well" + +(Notice that next log metadata would need to be signed by the current shard key.) + +(Notice that it is not sufficient to say that monitors and witnesses should both just +check for new shards periodically. This could lead to cases where all witnesses find +a new log to start cosigning, while an isolated monitor did not. Hence the above.) + +Pro: log operator can reuse their public key if there is no indication of compromise. +A verifier would simply not need to care about logging happening in a different shard. +Pro: witnesses and monitors have reliable automatic upgrade paths in the normal case. + +There are obviously more details to be discussed, the above captures an idea. + +How witnesses and monitors recover after a log's key was compromised? This is related +to how witnesses and monitors add new log operators. Transparent mailing list? + +Would it be a good idea to do some kind of cross-signing, so that a verifier with the +old key can still pick-up the log operator while getting security from witnessing? + +(Another related TODO to think about: would it be better to define shard_start as the +latest point in time that the log will be in a read+write mode? It might be sooner. +And another one. Is it possible/good to leave out shard_end with this approach?) |