On Wed, Jun 22, 2022 at 12:29:59AM +0000, brian m. carlson wrote: > > We've since migrated our default hash function from SHA-1 to SHA-1DC > > (except on vanilla OSX, see [2]). It's a variant SHA-1 that detects the > > SHAttered attack implemented by the same researchers. I'm not aware of a > > current viable SHA-1 collision against the variant of SHA-1 that we > > actually use these days. > > That's true, but that still doesn't let you store the data. There is > some data that you can't store in a SHA-1 repository, and SHA-1DC is > extremely slow. Using SHA-256 can make things like indexing packs > substantially faster. I'm curious if you have numbers on this. I naively converted linux.git to sha256 by doing "fast-export | fast-import" (the latter in a sha256 repo, of course, and then both repacked with "-f --window=250" to get reasonable apples-to-apples packs). Running "index-pack --verify" on the result takes about the same time (this is on an 8-core system, hence the real/user differences): [sha1dc] real 2m43.754s user 10m52.452s sys 0m36.745s [sha256] real 2m41.884s user 12m23.344s sys 0m35.222s The sha256 repo actually has about 10% fewer objects (I didn't investigate, but this is perhaps due to cutting out tags and a few other things to convince fast-export to finish running). I'm not sure about the extra user time (multicore timings here are funny because of frequency scaling, so I think the "real" line is more interesting). So sha256 actually comes out a bit worse here. On the other hand, this is just using our blk_SHA256 implementation. There may be faster alternatives (including ones with hardware support). I wouldn't be at all surprised if the difference isn't substantial in the long run, though. The repo is on the order of 100GB of object data. That's a lot to hash, but it's also just a lot to deal with at all (zlib inflating, applying deltas, etc). Anyway, this is a pretty rough cut at an experiment. I was mostly curious if you had done something more advanced, and/or gotten different results. -Peff
