This series is based on ds/multi-pack-index and jk/for-each-object-iteration. The multi-pack-index indexes objects across multiple pack-files. To speed up object lookups and abbreviations, we do not place the pack- files covered by the multi-pack-index into the packed_git linked list or the packed_git_mru list. Existing test coverage focused on typical uses and the main consumers of the multi-pack-index. To better understand the implications of the multi-pack-index with other scenarios, I ran the test suite after adding a step to 'git repack' to write a multi-pack-index, and to default core.multiPackIndex to 'true'. This commit is available as [1]. The following issues were discovered, and are fixed by this series: 1. The multi-pack-index did not distinguish between local and non-local pack-files. 2. A bad packed object was not inspected by object lookups in the multi- pack-index, so would loop infinitely trying to load the same object. 3. 'git count-objects --verbose' would not see the objects in the multi- pack-index and would report the multi-pack-index as garbage. 4. If the local object directory had a multi-pack-index but an alternate did not, then the multi-pack-index would be dropped. 5. If the multi-pack-index covered a pack-file that was paired with a reachability bitmap, then that bitmap would not be loaded. Several issues were resolved simply by making a new 'all_packs' list in the object store and replacing get_packed_git() calls with get_all_packs() calls. The all_packs list is a linked list that starts with the pack-files in multi-pack-indexes and then continues along the packed_git linked list. Also: I simplified the usage reports in 'git multi-pack-index' to help users who are entering parameters incorrectly. [1] https://github.com/derrickstolee/git/commit/098dd1d515b592fb165a276241d7d68d1cde0036 DO-NOT-MERGE: compute multi-pack-index on repack. I will send this commit as a separate patch so we can see the change I made and the one test I needed to fix (because it moves a pack-file, thereby making the multi-pack-index invalid). [2] https://github.com/derrickstolee/git/pull/9 A GitHub pull request containing this series. Derrick Stolee (9): multi-pack-index: provide more helpful usage info multi-pack-index: store local property midx: mark bad packed objects midx: stop reporting garbage midx: fix bug that skips midx with alternates packfile: add all_packs list treewide: use get_all_packs midx: test a few commands that use get_all_packs pack-objects: consider packs in multi-pack-index builtin/count-objects.c | 2 +- builtin/fsck.c | 4 ++-- builtin/gc.c | 4 ++-- builtin/multi-pack-index.c | 16 +++++++------- builtin/pack-objects.c | 42 +++++++++++++++++++++++++++++------ builtin/pack-redundant.c | 4 ++-- fast-import.c | 4 ++-- http-backend.c | 4 ++-- midx.c | 32 ++++++++++++++++++--------- midx.h | 7 ++++-- object-store.h | 6 +++++ pack-bitmap.c | 2 +- pack-objects.c | 2 +- packfile.c | 40 ++++++++++++++++++++++++++++----- packfile.h | 1 + server-info.c | 4 ++-- t/helper/test-read-midx.c | 2 +- t/t5319-multi-pack-index.sh | 44 ++++++++++++++++++++++++++++++++++--- 18 files changed, 168 insertions(+), 52 deletions(-) -- 2.18.0.118.gd4f65b8d14
