On Tue, Sep 24, 2024 at 01:32:01PM -0400, Taylor Blau wrote:
> This version is another fairly substantial reroll, with the main
> differences being renaming the "fast" SHA-1 options to "unsafe", as well
> as not running the collision checks via finalize_object_file() when
> handling loose objects (see the relevant patches for details on why).
I left some rambling comments on the collision check changes. Nothing
earth-shattering, but you may or may not want to tweak based on what I
said.
I'm happy with the s/fast/unsafe/ rename.
The rest of it was more or less the same, and looks good to me. I do
think with the extra collision-check it would be OK to teach index-pack
to use the fast sha1, too. That would speed up receiving objects by the
same absolute numbers (but less as a relative portion, since delta
resolution is usually much more expensive). And it would also make
verify-pack a bit faster, too (IIRC fsck for some reason does not use
verify-pack, so its code is a potential candidate, too, if it's not
already using csum-file).
Those don't strictly need to come now, but it seems like they might be
worthwhile while we're in the area (OTOH, using the fast hash only when
sending is a belt-and-suspenders with the collision check).
> Note also there is an important bug fix in finalize_object_file() to
> unlink() the temporary file when we do run the collision check, but no
> collisions were found. This bug was causing a pile-up of tmp_obj_XXXXXX
> files in GitHub's infrastructure.
Oops. :) I wondered if we could have a test here, but I don't think this
can be easily triggered in the tests. The loose object collision happens
only via TOCTOU race. I also tried instrumenting the code like below,
and it triggers zero times in the test suite.
diff --git a/object-file.c b/object-file.c
index b9a3a1f62d..d9172df8d9 100644
--- a/object-file.c
+++ b/object-file.c
@@ -40,6 +40,7 @@
#include "fsck.h"
#include "loose.h"
#include "object-file-convert.h"
+#include "trace.h"
/* The maximum size for an object header. */
#define MAX_HEADER_LEN 32
@@ -1994,6 +1995,7 @@ int finalize_object_file(const char *tmpfile, const char *filename)
int finalize_object_file_flags(const char *tmpfile, const char *filename,
enum finalize_object_file_flags flags)
{
+ static struct trace_key t = TRACE_KEY_INIT(COLLISION);
struct stat st;
int ret = 0;
@@ -2031,6 +2033,8 @@ int finalize_object_file_flags(const char *tmpfile, const char *filename,
errno = saved_errno;
return error_errno(_("unable to write file %s"), filename);
}
+ if (flags & FOF_SKIP_COLLISION_CHECK)
+ trace_printf_key(&t, "skipping check of %s and %s", tmpfile, filename);
if (!(flags & FOF_SKIP_COLLISION_CHECK) &&
check_collision(tmpfile, filename))
return -1;
