This series resolves a bug that was reported[1] by Johannes, and
investigated by him, Abhradeep, and Stolee in that same sub-thread.
The crux of the issue is that a MIDX bitmap can enter a corrupt state
when changing the preferred pack from its value in an existing MIDX in
certain circumstances as described in the first and final patches.
This series is structured as follows:
- The first patch of this series adds a test which demonstrates the
problem. (This is an improvement from the debugging in [1], where we
only noticed the problem racily in an existing test, and only in
SHA-256 mode).
- The next small handful of patches refactor midx.c's
`get_sorted_entries()` function to make fixing this bug more
straightforward.
- The final patch resolves the bug and updates the test to no longer
expect failure.
A couple of meta-notes:
- This bug has existed since the introduction of MIDX bitmaps, but
probably wasn't noticed until now since it is only triggerable when
the `--stdin-packs` mode *isn't* passed, so it never occurs when
invoked via `git repack`.
- We could likely change the behavior of 56d863e979 (midx: expose
`write_midx_file_only()` publicly, 2021-09-28), which explicitly
disables reusing the existing MIDX (by avoiding loading it
altogether) when `--stdin-packs` is given.
The rationale in the comment added by 56d863e979 is somewhat unclear,
but I have a vague recollection of running into a bug that was
squashed by avoiding reusing an existing MIDX to write one with
bitmaps. This was likely that bug.
Thanks in advance for your review, and thanks to Johannes, Abhradeep,
and Stolee for investigating this bug while I was on vacation.
[1]: https://lore.kernel.org/git/p3r70610-8n52-s8q0-n641-onp4ps01330n@xxxxxx/
Taylor Blau (6):
t5326: demonstrate potential bitmap corruption
t/lib-bitmap.sh: avoid silencing stderr
midx.c: extract `struct midx_fanout`
midx.c: extract `midx_fanout_add_midx_fanout()`
midx.c: extract `midx_fanout_add_pack_fanout()`
midx.c: include preferred pack correctly with existing MIDX
midx.c | 128 ++++++++++++++++++++++------------
t/lib-bitmap.sh | 2 +-
t/t5326-multi-pack-bitmaps.sh | 43 ++++++++++++
3 files changed, 127 insertions(+), 46 deletions(-)
--
2.37.0.1.g1379af2e9d
