The multi-pack-index file provides faster lookups in repos with many packfiles by duplicating the information from multiple pack-indexes into a single file. This series allows us to verify a multi-pack-index using 'git multi-pack-index verify' and 'git fsck' (when core.multiPackIndex is true). The pattern for the tests is similar to that found in t5318-commit-graph.sh. During testing, I found a bug in how we check for the size of off_t (we are not actually checking off_t, but instead uint32_t). See "multi-pack-index: fix 32-bit vs 64-bit size check". Thanks to Ævar [1], I added a commit that provides progress updates when checking object offsets. Based on ds/multi-pack-index [1] https://public-inbox.org/git/20180904202729.13900-1-avarab@xxxxxxxxx/T/#u Derrick Stolee (11): multi-pack-index: add 'verify' verb multi-pack-index: verify bad header multi-pack-index: verify corrupt chunk lookup table multi-pack-index: verify packname order multi-pack-index: verify missing pack multi-pack-index: verify oid fanout order multi-pack-index: verify oid lookup order multi-pack-index: fix 32-bit vs 64-bit size check multi-pack-index: verify object offsets multi-pack-index: report progress during 'verify' fsck: verify multi-pack-index Documentation/git-multi-pack-index.txt | 10 ++ builtin/fsck.c | 18 ++++ builtin/multi-pack-index.c | 4 +- midx.c | 113 ++++++++++++++++---- midx.h | 1 + t/t5319-multi-pack-index.sh | 136 ++++++++++++++++++++++++- 6 files changed, 262 insertions(+), 20 deletions(-) base-commit: 6a22d521260f86dff8fe6f23ab329cebb62ba4f0 Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-34%2Fderrickstolee%2Fmidx%2Fverify-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-34/derrickstolee/midx/verify-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/34 Range-diff vs v1: 1: 8dc38afe2b ! 1: d8ffd84d67 multi-pack-index: add 'verify' verb @@ -47,7 +47,7 @@ static char const * const builtin_multi_pack_index_usage[] = { - N_("git multi-pack-index [--object-dir=<dir>] write"), -+ N_("git multi-pack-index [--object-dir=<dir>] [write|verify]"), ++ N_("git multi-pack-index [--object-dir=<dir>] (write|verify)"), NULL }; 2: 787e1fb616 ! 2: 9590895830 multi-pack-index: verify bad header @@ -61,10 +61,10 @@ +# usage: corrupt_midx_and_verify <pos> <data> <objdir> <string> +corrupt_midx_and_verify() { -+ POS=$1 -+ DATA="${2:-\0}" -+ OBJDIR=$3 -+ GREPSTR="$4" ++ POS=$1 && ++ DATA="${2:-\0}" && ++ OBJDIR=$3 && ++ GREPSTR="$4" && + FILE=$OBJDIR/pack/multi-pack-index && + chmod a+w $FILE && + test_when_finished mv midx-backup $FILE && 3: b385aa2abf = 3: 2448173844 multi-pack-index: verify corrupt chunk lookup table 4: 37ee24c82b = 4: 947241bfdc multi-pack-index: verify packname order 5: b747da415c = 5: 4058867380 multi-pack-index: verify missing pack 6: 58e5c09468 = 6: ea1c522702 multi-pack-index: verify oid fanout order 7: b21772d054 = 7: 511791de91 multi-pack-index: verify oid lookup order 8: b08d3f0055 = 8: 210649bf83 multi-pack-index: fix 32-bit vs 64-bit size check 9: e1498aea45 ! 9: ef20193d59 multi-pack-index: verify object offsets @@ -21,7 +21,8 @@ if (pack_int_id >= m->num_packs) - BUG("bad pack-int-id"); -+ die(_("bad pack-int-id")); ++ die(_("bad pack-int-id: %u (%u total packs"), ++ pack_int_id, m->num_packs); if (m->packs[pack_int_id]) return 0; 10: acf8cfd632 = 10: 29ebc17161 multi-pack-index: report progress during 'verify' 11: 09d16aff20 ! 11: 406c88b456 fsck: verify multi-pack-index @@ -40,14 +40,14 @@ --- a/t/t5319-multi-pack-index.sh +++ b/t/t5319-multi-pack-index.sh @@ - DATA="${2:-\0}" - OBJDIR=$3 - GREPSTR="$4" -+ COMMAND="$5" + DATA="${2:-\0}" && + OBJDIR=$3 && + GREPSTR="$4" && ++ COMMAND="$5" && + if test -z "$COMMAND" + then + COMMAND="git multi-pack-index verify --object-dir=$OBJDIR" -+ fi ++ fi && FILE=$OBJDIR/pack/multi-pack-index && chmod a+w $FILE && test_when_finished mv midx-backup $FILE && -- gitgitgadget
