I'm sure this won't cause anyone any headaches:
"Since DEFLATE is the underlying compression used in archive/zip, compress/gzip, compress/zlib, and image/png, the outputs from those packages may also have changed."
14
Replies
4
Boosts
@andrewnez
I hope no software forge uses go to generate git-archive tarballs on the fly, that are later used by distros to build packages,and checked against a known-good hash...
@wolf480pl @andrewnez archive tarballs should be among the releases' files
@SRAZKVT
some upstreams (eg. bfdd) don't have releases, so some distro's (eg. OpenWrt) packages use the "generate tarball from commit" endpoint as a source download url
@andrewnez
@wolf480pl @SRAZKVT @andrewnez
Huh, they could use git clone --depth 1 and diff the contents instead of relying on tarball integrity.
This keeps coming up again and again.
Commit hashes ought to remain stable.
@CyReVolt @wolf480pl @andrewnez a git clone and a digest of the tree should suffice, but i don't know of a message digest for filesystem trees
maybe @hipsterelectron ?
@SRAZKVT @CyReVolt @wolf480pl @andrewnez this is a fascinating topic and i believe deserves a new checksum algorithm to minimize recomputation upon change. in particular all the SHAs are actively designed against this since they are used in cryptography to hide information and cannot be computed in parallel (hence requiring hacks like sorting directory entries). hash bucketing entries by name is a good start
@hipsterelectron
like, I think git already does this, in a way where recomputation after a single-file update is O(updated file size + diectory depth)
If that method doesn't solve your problem then I don't understand what problem you're trying to solve.
@SRAZKVT @CyReVolt @andrewnez
@wolf480pl that's cool i wasn't asking for help though
@hipsterelectron
But but like, let's bring it back on topic of the discussion we were having.
If directory hashing as done by git is not sufficient for what you and @ SRAZKVT expect for verifying an upstream source archive, I'm curious what other properties you'd need that git tree hashes don't have, and which I'm not seeing.
@wolf480pl @hipsterelectron my issue with git is it uses sha1 (yes, i know git technically supports sha256 but basically no one is using that)
@SRAZKVT
Ok but you can just reimplement the same scheme with sha256, Keccak or Blake2 instead and it will retain all of its nice properties.
Yeah, it's not something available out of the box, and not something that will match existing hash values people are already using, but it doesn't require new cryptography.
@hipsterelectron
@wolf480pl @SRAZKVT all of the things you mentioned here are examples of new cryptography. they are specifically intended for use as cryptographic message digests which means indifferentiability. this means they can be used for signatures without leaking information about the input. in the case of a checksum mismatch for source code, we don't want that at all: we want the forensic capacity to determine what changed. this is also something we want for filesystem integrity
@hipsterelectron
they're existing cryptography that has widespread Implementations and could be used as a drop-in replecement in higher-level schemes, such as git's directory hashing scheme.
Forensic capacity to determine what's changed sounds interesting, but is it possible information-theoretically, without having the plaintext of both archives?
