On 2019-12-22 at 06:41:33, Michael Clark wrote: > Recommendations > =============== > > After performing multiple benchmarks and thinking about this for > several weeks I have come to the conclusion that SHA512/256 is a > very good alternative to SHA256, both from the perspective of its > length extension resistance, and performance. It seems that NewHash > implementation is some way off so it is not too late to change. > > - Consider SHA512/256 for NewHash This has been discussed extensively on the list. Git is not in general vulnerable to length extension attacks because it hashes the length of the object as part of the object header. This was documented as one of the known aspects in the transition plan in Documentation/technical/hash-function-transition.txt. We've considered a lot of hash algorithms, including BLAKE2b-256, SHA-256, SHA-512/256, SHA-3-256, SHAKE256 with a 256-bit length, and others. This was no doubt controversial, but we discussed the pros and cons of many options and this is what we decided. I will state that while our choice was not one of my more preferred options, I value that we have a consensus on the way forward. Among the reasons we chose SHA-256 is performance on newer x86-64 processors with the SHA extensions, which support only SHA-1 and SHA-256, and compatibility with a wide variety of cryptographic libraries. I have no desire for Git to provide assembly implementations of NewHash, and that means people must rely on existing cryptographic libraries. Cryptographic libraries overwhelmingly do not support SHA-512/256. While OpenSSL does, it's only in the latest versions, and people generally cannot legally distribute both Git and OpenSSL linked together, so they must rely on system libraries for good performance. Apple Common Crypto, for example, does not document any SHA-512/256 implementations. There is also a benefit to chaining using a single algorithm for signatures, which therefore necessitates using SHA-256 or one of the other original SHA-2 algorithms until the next version of OpenPGP is implemented which supports SHA-3. Because we decided some time ago, I've sent in a bunch of patches to our testsuite to make it work with SHA-256. Some of these patches are general, in that they make the tests generate values which are used, or they are specific to the length of the hash algorithm. Others use specific hash values, and changing the hash algorithm will require recomputing all of these values. Absent a compelling security reason to abandon SHA-256, such as a significant previously unknown cryptographic weakness, I don't plan to reimplement all of this work. Updating our testsuite to work successfully with SHA-256 has taken a huge amount of time, and this work has been entirely done on my own free time because I want the Git project to be successful. That doesn't even include the contributions of others who have reviewed, discussed, and contributed to the current work and transition plan. While I appreciate the benefits of alternatives to SHA-256, I'm a bit annoyed that you're proposing to throw away all of this work after we explicitly sought to build consensus about an algorithm so that this stage of the work could begin. Of course, nothing prevents you from proposing a new algorithm and fixing the testsuite to make it work, but I don't believe that we're likely to adopt a different algorithm at this point, given the previous discussion on the list. The code is designed to be pluggable to support arbitrary algorithms, but there are pieces which assume that the algorithm can be distinguished by length (e.g., the dumb HTTP protocol) and therefore you'd need to devise a different way to distinguish these versions. If you'd like to see what a Git binary that supports multiple hash algorithms looks like at the moment, you're welcome to look at my transition-stage-4 branch at https://github.com/bk2204/git.git so you can get a fuller understanding of what's involved. You can run the testsuite with that code and GIT_TEST_DEFAULT_HASH=<hashname> to see what passes and fails. > SHA algorithm patches > ===================== > > I will be sending a patch series which adds SHA Algorithms which > are in the sha-algorithms branch of my git tree. > > - https://github.com/michaeljclark/git.git > - https://github.com/michaeljclark/git/tree/sha-algorithms/sha > > This is a precursor to asking some questions about the proposed > implementation of multiple or compat hashs (N-hash or 2-hash?). > I am curious about the pack format and directory structure for > loose objects. It seems there needs to be a single hash chosen > for loose object filenames, and auxilliary hashes will need > hash to hash map to go from a compat hash to new hash. That's correct. There's documentation in the transition plan about what this will look like. A stage 4 implementation (where repositories are either SHA-1 only or SHA-256 only) has been written and I'll send it out once the remaining test fixes are merged. I should point out that we specifically chose not to support a plethora of algorithms because the goal is to encourage everybody to adopt a single, secure algorithm. The network protocol will need to interoperate using a single algorithm, which means the repository will need to contain a mapping for that algorithm. Since the current plan is to support no more than two algorithms at a time, one of which will need to be SHA-1, allowing users to pick and choose from a variety of algorithms harms interoperability. Seamless interoperability between SHA-1 and NewHash repositories was considered extremely important in the transition plan. That won't be coming immediately with the stage 4 implementation, but it is planned for the future. My hope is that whatever we pick now will be secure for some time to come and that when we need to adopt a newer hash, whether due to cryptographic advance or post-quantum cryptography, there will be no existing SHA-1 repositories and transition can take place from NewHash to NewNewHash. -- brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204
Attachment:
signature.asc
Description: PGP signature
