how on earth does openssh not only have incredibly fucked versioning but is the only package i've ever seen to mention a "privilege separation" chroot path which actively breaks non-root builds. are you fucking kidding me bruh. "privilege separation chroot" defaulting to /var/empty and it immediately errors if it can't create it????? why are your version strings like that bruh?????
like when you allocate more elements the conventional wisdom is that this reduces collisions as if the discrete bits are being stretched apart. but i also seem to recall that rand() % N is not uniform over [1,N] (i think bc it favors earlier elements?)
the second conventional wisdom which i believe i recall reading specifically online (likely more than once) regarding hash function implementation tips the first thing people would say "a modulus is an example of a hash function"
@hipsterelectron few weeks ago i went "how could i make a CAS collision free" until i realised that a hash function having a fixed size return is only useful to have comparisons be in constant time, but that if you don't need that you can use anything that gives any size
@hipsterelectron compression could be a reasonable hash function, but when in use in a self referencing CAS, the hashes would just get bigger and bigger and bigger as length of chains increase
@SRAZKVT so i think the standard "more space for fewer collisions" could be saved by the same solution for ranged randomness which iirc should work like ((float)rand() / (float)RAND_MAX) * ((float)N)
@SRAZKVT but i am also very curious about two further use cases:
- reusing an allocation for multiple stages in which the data distribution may be different
- constructing a representation of the occupied slots without translation into an index offset
(2) is obviously not going to produce constant-time lookups and i have no idea what it would be useful for but the idea of a fixed collision budget as a probabilistic measure should (i feel) mean we can mix that budget across the space in more than just this regimented way a * b * c
@SRAZKVT for a vcs my big realization was that cryptographic indifferentiability is not only wasted CPU but directly means that checksum comparison gives you only 1 bit of information,
and immediately i realized that progressive forensic reconstruction of modifications which produced the distinct top-level is VERY IMPORTANT for a VCS,
and then i realized it doesn't have to be "forensic", since we could say that is kind of the whole point
@hipsterelectron yea a checksum comparison is only one bit of information, that's the whole point of checksum comparison
@hipsterelectron honestly i still think a state based VCS is probably better for most as it is easier to think about (moving a file is more natural than producing / applying a change) while being easier to implement (necessary for redundancy)
@hipsterelectron for a VCS, a message digest to identify data objects is fundamentally flawed, as cryptographic hash functions are temporary on the scale of hundreds of years, and change is slow, and an incompatibility in the way we identify data objects would make two parts incapable of working together, which is something we don't want out of a collaborative VCS
@SRAZKVT so especially before i understood snapshots i was only able to think about the current state of the filesystem tree and minimizing checksum recomputation upon changes. the other big problem with checksums of directory trees (which is neither about indifferentiability, nor the interoperability goal you describe) is that if a single bit flips in a single file the entire dataset needs to be re-digested from scratch (i'm super disappointed zfs just dgaf about this at all)
@SRAZKVT https://keccak.team/files/TreeHashing.pdf keccak is SHA-3 whom i think basically read this paper and said "oh this is accessible enough for people outside the field to read we need to SEO ourselves next to tree hashing" because they wrote multiple papers saying "not only is keccak totally a tree hash but we made this other form kangaroo12 which is fast but less secure" https://keccak.team/files/KangarooTwelve.pdf
the first paper is really really good
@SRAZKVT i found it because BLAKE3 cites it as their single mention of any sort of correctness proof. and they leave out one of the soundness requirements. but the tree hashing paper is really really good bc it actively interrogates the structure of a digest function:
Most hash functions are constructed in a layered fashion. Traditionally, hash functions have variable input-length and fixed output-length. There is a mode of use that processes the input and in turn calls an underlying function F. Usually, this underlying function is
a compression function.
this is the wikipedia definition. unfortunately they say "compression" without defining what that means
In this section, we generalize this idea. We do not impose limits to the input- or output-length of the underlying function called the inner hash function and denoted by F .
so this is really interesting bc it undercuts a pretty important misconception. there was a terminology "N-bit security" popularized by NIST way way back when AES was new where they would characterize cryptographic systems in a really pseudoscientific way as like multiples of AES-256 bit security and say more bits = more security. this was probably not malicious but it was readily picked up as a shorthand by corporate vendors to swindle people.
one really important way this becomes misused as NIST gets gutted by the NSA is especially misinterpreting the standard buffer sizes used in NIST SHA standards like SHA-256/SHA-512 as if 512 is twice as secure. in fact, if your concern is not collision frequency but leaking secret data, 512 bits exposes more info. since digests are used for signatures, this can be used to leak your private key, if using djb's EdDSA which specifically ensures the key data prefixes the message data.
(btw, EdDSA uses SHA-512)
@SRAZKVT the reason we can keep extracting data indefinitely is because indifferentiability is kind of weird in that it limits how process outputs can be used to infer information about corresponding inputs. but the secret prefix attack on e.g. EdDSA indicates that indifferentiabillity is better understood as a negotiated standard among cryptographers because it maps to generalizable protocol guarantees without overly constraining the design space for implementations of the standard.
i think POSIX i/o is generally good, for similar reasons. the parts i will break are ones i do not believe users actively seek or that naturally map to code patterns or communication structures.
@SRAZKVT this part uses very confusing language:
Our generalization allows for dealing with hierarchical hash functions obtained by applying a tree hashing mode to an inner hash function that is itself sequential.
i thought at first this meant what i wanted to do with filesystem trees, so a SHA-256 hash of a file can be propagated into a hash of the directory tree safely. unfortunately by "message" they really do absolutely mean "totally ordered serial bit string"