how on earth does openssh not only have incredibly fucked versioning but is the only package i've ever seen to mention a "privilege separation" chroot path which actively breaks non-root builds. are you fucking kidding me bruh. "privilege separation chroot" defaulting to /var/empty and it immediately errors if it can't create it????? why are your version strings like that bruh?????
188
Replies
·
0
Boosts
@hipsterelectron for a VCS, a message digest to identify data objects is fundamentally flawed, as cryptographic hash functions are temporary on the scale of hundreds of years, and change is slow, and an incompatibility in the way we identify data objects would make two parts incapable of working together, which is something we don't want out of a collaborative VCS
@SRAZKVT so especially before i understood snapshots i was only able to think about the current state of the filesystem tree and minimizing checksum recomputation upon changes. the other big problem with checksums of directory trees (which is neither about indifferentiability, nor the interoperability goal you describe) is that if a single bit flips in a single file the entire dataset needs to be re-digested from scratch (i'm super disappointed zfs just dgaf about this at all)
@SRAZKVT https://keccak.team/files/TreeHashing.pdf keccak is SHA-3 whom i think basically read this paper and said "oh this is accessible enough for people outside the field to read we need to SEO ourselves next to tree hashing" because they wrote multiple papers saying "not only is keccak totally a tree hash but we made this other form kangaroo12 which is fast but less secure" https://keccak.team/files/KangarooTwelve.pdf
the first paper is really really good
@SRAZKVT i found it because BLAKE3 cites it as their single mention of any sort of correctness proof. and they leave out one of the soundness requirements. but the tree hashing paper is really really good bc it actively interrogates the structure of a digest function:
Most hash functions are constructed in a layered fashion. Traditionally, hash functions have variable input-length and fixed output-length. There is a mode of use that processes the input and in turn calls an underlying function F. Usually, this underlying function is
a compression function.
this is the wikipedia definition. unfortunately they say "compression" without defining what that means
In this section, we generalize this idea. We do not impose limits to the input- or output-length of the underlying function called the inner hash function and denoted by F .
so this is really interesting bc it undercuts a pretty important misconception. there was a terminology "N-bit security" popularized by NIST way way back when AES was new where they would characterize cryptographic systems in a really pseudoscientific way as like multiples of AES-256 bit security and say more bits = more security. this was probably not malicious but it was readily picked up as a shorthand by corporate vendors to swindle people.
one really important way this becomes misused as NIST gets gutted by the NSA is especially misinterpreting the standard buffer sizes used in NIST SHA standards like SHA-256/SHA-512 as if 512 is twice as secure. in fact, if your concern is not collision frequency but leaking secret data, 512 bits exposes more info. since digests are used for signatures, this can be used to leak your private key, if using djb's EdDSA which specifically ensures the key data prefixes the message data.
(btw, EdDSA uses SHA-512)
@SRAZKVT the reason we can keep extracting data indefinitely is because indifferentiability is kind of weird in that it limits how process outputs can be used to infer information about corresponding inputs. but the secret prefix attack on e.g. EdDSA indicates that indifferentiabillity is better understood as a negotiated standard among cryptographers because it maps to generalizable protocol guarantees without overly constraining the design space for implementations of the standard.
i think POSIX i/o is generally good, for similar reasons. the parts i will break are ones i do not believe users actively seek or that naturally map to code patterns or communication structures.
@SRAZKVT this part uses very confusing language:
Our generalization allows for dealing with hierarchical hash functions obtained by applying a tree hashing mode to an inner hash function that is itself sequential.
i thought at first this meant what i wanted to do with filesystem trees, so a SHA-256 hash of a file can be propagated into a hash of the directory tree safely. unfortunately by "message" they really do absolutely mean "totally ordered serial bit string"
@SRAZKVT trying to figure out the way to start this. the serial bit string is the standard assumption for parsing theory. streams of bytes are the lowest common denominator for IPC. it's very weak and very strong:
this assumption is very weak, because we can assume no grammatical structure beyond distinguishing beween a fixed-size language of bit/byte/token values.
- in a graphical model, this can be a vertex label, and a single unlabeled edge connecting each discrete input element can be used to represent the strict total ordering of data. this looks like a-b-3-1, for the string "ab31".
this assumption is very strong, because it can be used to identify a hamiltonian path (a path through all vertices exactly once, np-complete on general graphs) in constant time.
- that sounds like a meme, but it is why i consider parsing trivially parallelizable, as well as very amenable to coroutine formulations that not only let us codify forward progress and termination guarantees, but also fail to require "backtracking" whatsoever, as there are no "hidden dependencies".
generally, my parsing framework seeks to build up structured/labeled/weighted graphs, which can represent the arbitrary complexity of a complete graph, by compiling an input grammar (a user-specified graph) into a collection of data structures--most basically, by mapping all possible transitions between pairs of input tokens into a sequence of all possible edges in the input grammar graph they might represent. this transforms a-b-3-1 into something like 0-[x1]-1-[x2,x3,x5]-3-[x4]-1.
@SRAZKVT the input has been overlaid with all the potential complexity of the grammar, and the task is to find a path of edges that spans the whole input. you will note that this is no longer about switching upon the value of individual bytes at all. instead we are directly constructing a parse tree, and seeing if it fits.
why would anyone do such a thing? well, one thing we get for free is that expanding our input to the left or right never invalidates any previous work. this is why we can parallelize, but it also means we naturally support streaming input. this is an extremely curious achievement!
what happens if we mutate--invalidating data in the middle of our previous work? in fact, we lose nothing at all! we do have to identify paths through the new data (through the new pair-transition edges), but at no point have we made a decision which precludes anything other than the edgewise transitions between neighboring tokens--exactly the edges which are re-introduced at either side of each modified token.
@SRAZKVT so i referred to parsing, because the data is serialized.
to represent filesystem state with a checksum, we also want to minimize recomputation effort. but "minimize recomputation" is not all: we might also achieve a minimal encoding, for example.
@SRAZKVT all this line of thinking led me to believe, as you said above, that this is not a matter of checksums, but of data representation.
@SRAZKVT this is why i find the idea of something like radix bucketing or bloom filters so generative, because there is no unbounded monotonic negative incentive like collision reduction in a classical hash table (which is always improved by just allocating more memory = paying more money), but instead the ability to leverage anything you know that the compiler and libraries can't!
in particular, each bucket over a finite input domain corresponds to a relation (i.e. edge) across inputs (vertices). so, if you design your hash function not just to use the output bits to distinguish, but also to share features of the data to link together, you can use that to construct a graph.
@SRAZKVT that's fucked up. i don't even know what you can do with that. i just realized it
@SRAZKVT git lets you provide diff engines with a terrible config API, which i'm pretty sure is absolutely not used internally but just to display in certain user-facing commands. this is SUPER HILARIOUS because uhhhhhhhhhhh doesn't that mean the diff engine can just lie
@SRAZKVT you mentioned compression above--i believe compression of the canonical state is not what we're interested in per se but rather normalization
@hipsterelectron @SRAZKVT normalization brings to mind the oft-forgotten existence of smudge and clean filters which are arbitrary commands that you run to convert between the stored and working states of files.
