From: Son Luong Ngoc <sluongng@xxxxxxxxx>
Multi-Pack-Index repack is an incremental, repack solutions
that allows user to consolidate multiple packfiles in a non-disruptive
way. However the new packfile could be created without some of the
capabilities of a packfile that is created by calling `git repack`.
This is because with `git repack`, there are configuration that would
enable different flags to be passed down to `git pack-objects` plumbing.
In this patch, I applies those flags into `git multi-pack-index repack`
so that it respect the `repack.*` config series.
Note: I left out `repack.packKeptObjects` intentionally as I dont think
its relevant to midx repack use case.
Signed-off-by: Son Luong Ngoc <sluongng@xxxxxxxxx>
---
midx: apply gitconfig to midx repack
Midx repack has largely been used in Microsoft Scalar on the client side
to optimize the repository multiple packs state. However when I tried to
apply this onto the server-side, I realized that there are certain
features that were lacking compare to git repack. Most of these features
are highly desirable on the server-side to create the most optimized
pack possible.
One of the example is delta_base_offset, comparing an midx repack
with/without delta_base_offset, we can observe significant size
differences.
> du objects/pack/*pack
14536 objects/pack/pack-08a017b424534c88191addda1aa5dd6f24bf7a29.pack
9435280 objects/pack/pack-8829c53ad1dca02e7311f8e5b404962ab242e8f1.pack
Latest 2.26.2 (without delta_base_offset)
> git multi-pack-index write
> git multi-pack-index repack
> git multi-pack-index expire
> du objects/pack/*pack
9446096 objects/pack/pack-366c75e2c2f987b9836d3bf0bf5e4a54b6975036.pack
With delta_base_offset
> git version
git version 2.26.2.672.g232c24e857.dirty
> git multi-pack-index write
> git multi-pack-index repack
> git multi-pack-index expire
> du objects/pack/*pack
9152512 objects/pack/pack-3bc8c1ec496ab95d26875f8367ff6807081e9e7d.pack
In this patch, I intentionally leaving out repack.packKeptObjects as I
don't think its very relevant to midx repack use case:
* One could always exclude biggest packs with --batch-size option
* For non-biggest-packs exclusion use case, its rather rare (unless you
want to have a special pack with only commits and trees being
excluded from repack to serve partial clone better?)
Please let me know if anyone think that we should include that option
for the sake of completions.
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-626%2Fsluongng%2Fsluongngoc%2Fmidx-config-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-626/sluongng/sluongngoc/midx-config-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/626
midx.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/midx.c b/midx.c
index 9a61d3b37d9..88f16594268 100644
--- a/midx.c
+++ b/midx.c
@@ -1361,6 +1361,10 @@ static int fill_included_packs_batch(struct repository *r,
return 0;
}
+static int delta_base_offset = 1;
+static int write_bitmaps = -1;
+static int use_delta_islands;
+
int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags)
{
int result = 0;
@@ -1381,12 +1385,25 @@ int midx_repack(struct repository *r, const char *object_dir, size_t batch_size,
} else if (fill_included_packs_all(m, include_pack))
goto cleanup;
+ git_config_get_bool("repack.usedeltabaseoffset", &delta_base_offset);
+ git_config_get_bool("repack.writebitmaps", &write_bitmaps);
+ git_config_get_bool("repack.usedeltaislands", &use_delta_islands);
+
argv_array_push(&cmd.args, "pack-objects");
strbuf_addstr(&base_name, object_dir);
strbuf_addstr(&base_name, "/pack/pack");
argv_array_push(&cmd.args, base_name.buf);
+ if (delta_base_offset)
+ argv_array_push(&cmd.args, "--delta-base-offset");
+ if (use_delta_islands)
+ argv_array_push(&cmd.args, "--delta-islands");
+ if (write_bitmaps > 0)
+ argv_array_push(&cmd.args, "--write-bitmap-index");
+ else if (write_bitmaps < 0)
+ argv_array_push(&cmd.args, "--write-bitmap-index-quiet");
+
if (flags & MIDX_PROGRESS)
argv_array_push(&cmd.args, "--progress");
else
base-commit: b34789c0b0d3b137f0bb516b417bd8d75e0cb306
--
gitgitgadget
