On Wed, Aug 10, 2022 at 2:50 PM Johannes Schindelin
<Johannes.Schindelin@xxxxxx> wrote:
>
> Hi Abhradeep,
> I instrumented this, and saw that the `multi-pack-index` and
> `multi-pack-index*.bitmap` files were unchanged by the `git repack`
> invocation.
Yeah, those two files remain unchanged here.
> Re-generating the MIDX bitmap forcefully after the repack seems to fix
> things over here:
>
> -- snip --
> diff --git a/t/lib-bitmap.sh b/t/lib-bitmap.sh
> index a95537e759b..564124bda27 100644
> --- a/t/lib-bitmap.sh
> +++ b/t/lib-bitmap.sh
> @@ -438,7 +438,10 @@ midx_bitmap_partial_tests () {
>
> test_expect_success 'setup partial bitmaps' '
> test_commit packed &&
> +ls -l .git/objects/pack/ &&
> git repack &&
> +git multi-pack-index write --bitmap &&
> +ls -l .git/objects/pack/ &&
> test_commit loose &&
> git multi-pack-index write --bitmap 2>err &&
> test_path_is_file $midx &&
> -- snap --
>
> This suggests to me that the `multi-pack-index write --bitmap 2>err` call
> in this hunk might reuse a stale MIDX bitmap, and that _that_ might be
> the root cause of this breakage.
Yeah, the `multi-pack-index write --bitmap 2>err` is creating the
problem. More specifically the `multi-pack-index write` part. As you
can see in my previous comment (if you get the comment), I shared a
screenshot there which pointed out that the multi-pack-index files in
both cases are different. The portion from which it started to differ
belongs to the `RIDX` chunk.
So, I used some debug lines in `midx_pack_order()` function[1] and
found that the objects are sorted differently in those cases (i.e.
passing case and failing case). For passing case, the RIDX chunk
contents are like below -
pack_order = [ 1, 36, 11, 6, 18, 3, 19, 12, 5, 31, 27, 23, 29, 8, 38,
22, 9, 15, 14, 24, 37, 28, 7, 39, 10, 34, 26, 4, 30, 33, 2, 35, 17,
32, 0, 21, 16, 25, 13, 40, 20,]
And in the failing case, this is -
pack_order = [ 12, 18, 3, 19, 1, 36, 11, 6, 5, 31, 27, 23, 29, 8, 38,
22, 9, 15, 14, 24, 37, 28, 7, 39, 10, 34, 26, 4, 30, 33, 2, 35, 17,
32, 0, 21, 16, 25, 13, 40, 20,]
I went further and realized that this is due to the line[2] -
if (!e->preferred)
data[i].pack |= (1U << 31);
I.e. 4- 5 `pack_midx_entry` objects have different `preferred` values
in those cases. For example,
"46193a971f5045cb3ca6022957541f9ccddfbfe78591d8506e2d952f8113059b"
(with pack order 12) is `preferred` in failing case (that's why it is
in the first position) and the same is `not preferred` in the passing
case.
It may be because of reusing a stale midx bitmap (as you said). But I
am not sure. Just to ensure myself, I compared all the other
packfiles, idx files and a pack `.bitmap` file (which you can see
using ls command) of failing and passing cases and found that they are
the same.
Thanks :)
[1] https://github.com/git/git/blob/c50926e1f48891e2671e1830dbcd2912a4563450/midx.c#L861
[2] https://github.com/git/git/blob/c50926e1f48891e2671e1830dbcd2912a4563450/midx.c#L872
