If Chris still has that huge .pack & .idx pair, it would be a very good guinea pig to try a few things on, assuming that this problem is that the pack-check.c feeds a huge blob to SHA-1 function with a single call.
I do not still have it, but I can pretty easily regenerate it. Should have it again in another nine hours or so. :)
(1) Apply the attached patch on top of "master" (the patch should apply to 1.4.1 almost cleanly as well, except that we have hashcmp(a,b) instead of memcmp(a,b,20) since then), and see what it says about the packfile. If your suspicion is correct, it should complain about your SHA-1 implementation. (2) Try tip of "next" to see if its verify-pack passes the check. Again, if your suspicion is correct, it should, since it uses Shawn's sliding mmap() stuff that will not feed the whole pack in one go. (3) I suspect that the tip of "master" should work except verify-pack. It may be interesting to see how well the tip of "master" and "next" performs on the resulting huge pack (say, "time git log -p HEAD >/dev/null"). I am hoping this would be another datapoint to judge the runtime penalty of Shawn's sliding mmap() in "next" -- I suspect the penalty is either negligible or even negative.
I'll try all of this after the pack is regenerated. Thanks! - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html