I'm sure this won't cause anyone any headaches:
"Since DEFLATE is the underlying compression used in archive/zip, compress/gzip, compress/zlib, and image/png, the outputs from those packages may also have changed."
10
Replies
2
Boosts
@andrewnez
I hope no software forge uses go to generate git-archive tarballs on the fly, that are later used by distros to build packages,and checked against a known-good hash...
@wolf480pl @andrewnez archive tarballs should be among the releases' files
@SRAZKVT
some upstreams (eg. bfdd) don't have releases, so some distro's (eg. OpenWrt) packages use the "generate tarball from commit" endpoint as a source download url
@andrewnez
@wolf480pl @SRAZKVT @andrewnez
Huh, they could use git clone --depth 1 and diff the contents instead of relying on tarball integrity.
This keeps coming up again and again.
Commit hashes ought to remain stable.
@CyReVolt @wolf480pl @andrewnez a git clone and a digest of the tree should suffice, but i don't know of a message digest for filesystem trees
maybe @hipsterelectron ?
@SRAZKVT @CyReVolt @wolf480pl @andrewnez this is a fascinating topic and i believe deserves a new checksum algorithm to minimize recomputation upon change. in particular all the SHAs are actively designed against this since they are used in cryptography to hide information and cannot be computed in parallel (hence requiring hacks like sorting directory entries). hash bucketing entries by name is a good start
@hipsterelectron @SRAZKVT @CyReVolt @wolf480pl the go proxy sumdb uses dirhash to avoid this problem: https://pkg.go.dev/golang.org/x/mod/sumdb/dirhash
@andrewnez @SRAZKVT @CyReVolt @wolf480pl that's not avoiding the problem at all, it's erasing information that makes it much easier for an attacker to generate hash collisions
@hipsterelectron @andrewnez @SRAZKVT @wolf480pl
Last time I checked, SHA256 wasn't easy to find collisions for. Has there been progress? Otherwise, I think it's very unlikely giving an advantage.
In that note, the hash of a tarball would be equally prone to collision attacks. Some distros use multi-hashes for that reason, i.e., add another hash using another algorithm/family.
@CyReVolt @andrewnez @SRAZKVT @wolf480pl it is still subject to several attacks common to all merkle-damgård constructions https://en.wikipedia.org/wiki/Merkle%E2%80%93Damg%C3%A5rd_construction#Security_characteristics generally, treating a fixed-length file the same way as an indefinite-length byte stream is a bad idea (length extension is the biggest problem, although that doesn't affect checksum-based integrity). cryptographic checksums intentionally seek to achieve a notion of indifferentiability which makes modifications opaque and is actively not what you want for open source code that is not secret
