On Wed, Aug 08, 2018 at 10:45:49PM -0400, Paul Smith wrote:
> On Wed, 2018-08-08 at 14:24 -0400, Jeff King wrote:
> > If so, can you try running it under gdb and getting a stack trace?
> > Something like:
> >
> > gdb git
> > [and then inside gdb...]
> > set args pack-objects --all --reflog --indexed-objects foo </dev/null
> > break die
> > run
> > bt
> >
> > That might give us a clue where the broken object reference is coming
>
> Here we go. I can rebuild with -Og or -O0 if more detailed debugging
> is needed; most everything appears to be optimized out:
No, I think this is enough to give a general sense of the problem
location.
> Compressing objects: 100% (107777/107777), done.
> Writing objects: 54% (274416/508176)
> Thread 1 "git" hit Breakpoint 1, die (err=err@entry=0x5a373a "unable to read %s") at usage.c:119
> 119 {
> (gdb) bt
> #0 die (err=err@entry=0x5a373a "unable to read %s") at usage.c:119
> #1 0x00000000004563f3 in get_delta (entry=<optimized out>) at builtin/pack-objects.c:143
> #2 write_no_reuse_object () at builtin/pack-objects.c:308
> #3 0x0000000000456592 in write_reuse_object (usable_delta=<optimized out>, limit=<optimized out>, entry=<optimized out>, f=<optimized out>) at builtin/pack-objects.c:516
> #4 write_object (write_offset=<optimized out>, entry=0x7fffc9a8d940, f=0x198fb70) at builtin/pack-objects.c:518
> #5 write_one () at builtin/pack-objects.c:576
> #6 0x00000000004592f0 in write_pack_file () at builtin/pack-objects.c:849
> #7 cmd_pack_objects (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin/pack-objects.c:3354
> #8 0x0000000000404f06 in run_builtin (argv=<optimized out>, argc=<optimized out>, p=<optimized out>) at git.c:417
> #9 handle_builtin (argc=<optimized out>, argv=<optimized out>) at git.c:632
> #10 0x0000000000405f21 in run_argv (argv=0x7fffffffe210, argcp=0x7fffffffe21c) at git.c:761
> #11 cmd_main (argc=<optimized out>, argc@entry=6, argv=<optimized out>, argv@entry=0x7fffffffe448) at git.c:761
> #12 0x0000000000404b15 in main (argc=6, argv=0x7fffffffe448) at common-main.c:45
So that's quite unexpected. I assumed we'd have hit this problem while
deciding _which_ objects to write. But we get all the way to the point
of writing out the result before we notice it's missing.
I don't think I've run such a case before, but I wonder if "pack-objects
--all" is too lax about adding missing blobs during its object traversal
(especially during the "unreachable but recent" part of the traversal
that I mentioned, which should silently omit missing objects). I played
around with recreating this situation, though, and I don't think it's
possible to cause the results you're seeing. We come up with a list of
recent objects, but we only use it as a look-up index for discarding
too-old objects. So:
- it wouldn't ever cause us to choose to write an object into a pack,
which is what you're seeing
- we'd never consider a missing object; it's a pure lookup table, and
the actual list of objects we consider is found by walking the set
of packs
So that's probably a dead end.
What I really wonder is where we found out about that object name in the
first place. Can you instrument your Git build like this:
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 71056d8294..5ff6de5ddf 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1112,6 +1112,13 @@ static int add_object_entry(const struct object_id *oid, enum object_type type,
struct packed_git *found_pack = NULL;
off_t found_offset = 0;
uint32_t index_pos;
+ static const struct object_id funny_oid = {
+ "\xc1\x04\xb8\xfb\x36\x31\xb5\xc5\x46\x95"
+ "\x20\x6b\x2f\x73\x31\x0c\x02\x3c\x99\x63"
+ };
+
+ if (!oidcmp(oid, &funny_oid))
+ warning("found funny oid");
display_progress(progress_state, ++nr_seen);
and similarly get a backtrace when we hit that warning()? (Or if you're
a gdb expert, you could probably use a conditional breakpoint, but I
find just modifying the source easier).
-Peff
