Here is another reroll of my series to implement "cruft packs". This is really
more like v4.1, since it has only cosmetic changes incorporated from the review
on v4 of this topic.
Since last time:
- The new section in pack-format.txt (describing the ".mtimes" format) now
says at the top "all 4-byte numbers are in network byte order", and avoids
repeating "network [byte] order" throughout that section to reduce
confusion.
- A sub-shell in t5329 which incorrectly masked over the exit code of a "git"
process was removed.
- An overly-long line in builtin/repack.c::collect_pack_filenames() was
eliminated, and matching braces are added.
...and that's pretty much it. In any case, a range-diff is included below.
Thanks again for all of the thoughtful feedback on this series.
Taylor Blau (17):
Documentation/technical: add cruft-packs.txt
pack-mtimes: support reading .mtimes files
pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles'
chunk-format.h: extract oid_version()
pack-mtimes: support writing pack .mtimes files
t/helper: add 'pack-mtimes' test-tool
builtin/pack-objects.c: return from create_object_entry()
builtin/pack-objects.c: --cruft without expiration
reachable: add options to add_unseen_recent_objects_to_traversal
reachable: report precise timestamps from objects in cruft packs
builtin/pack-objects.c: --cruft with expiration
builtin/repack.c: support generating a cruft pack
builtin/repack.c: allow configuring cruft pack generation
builtin/repack.c: use named flags for existing_packs
builtin/repack.c: add cruft packs to MIDX during geometric repack
builtin/gc.c: conditionally avoid pruning objects via loose
sha1-file.c: don't freshen cruft packs
Documentation/Makefile | 1 +
Documentation/config/gc.txt | 21 +-
Documentation/config/repack.txt | 9 +
Documentation/git-gc.txt | 5 +
Documentation/git-pack-objects.txt | 30 +
Documentation/git-repack.txt | 11 +
Documentation/technical/cruft-packs.txt | 123 ++++
Documentation/technical/pack-format.txt | 19 +
Makefile | 2 +
builtin/gc.c | 10 +-
builtin/pack-objects.c | 304 +++++++++-
builtin/repack.c | 185 +++++-
bulk-checkin.c | 2 +-
chunk-format.c | 12 +
chunk-format.h | 3 +
commit-graph.c | 18 +-
midx.c | 18 +-
object-file.c | 4 +-
object-store.h | 7 +-
pack-mtimes.c | 126 ++++
pack-mtimes.h | 15 +
pack-objects.c | 6 +
pack-objects.h | 25 +
pack-write.c | 93 ++-
pack.h | 4 +
packfile.c | 19 +-
reachable.c | 58 +-
reachable.h | 9 +-
t/helper/test-pack-mtimes.c | 56 ++
t/helper/test-tool.c | 1 +
t/helper/test-tool.h | 1 +
t/t5329-pack-objects-cruft.sh | 739 ++++++++++++++++++++++++
32 files changed, 1834 insertions(+), 102 deletions(-)
create mode 100644 Documentation/technical/cruft-packs.txt
create mode 100644 pack-mtimes.c
create mode 100644 pack-mtimes.h
create mode 100644 t/helper/test-pack-mtimes.c
create mode 100755 t/t5329-pack-objects-cruft.sh
Range-diff against v4:
-: ---------- > 1: f494ef7377 Documentation/technical: add cruft-packs.txt
1: 8f9fd21be9 ! 2: 91a9d21b0b pack-mtimes: support reading .mtimes files
@@ Documentation/technical/pack-format.txt: Pack file entry: <+
+== pack-*.mtimes files have the format:
+
++All 4-byte numbers are in network byte order.
++
+ - A 4-byte magic number '0x4d544d45' ('MTME').
+
+ - A 4-byte version identifier (= 1).
+
+ - A 4-byte hash function identifier (= 1 for SHA-1, 2 for SHA-256).
+
-+ - A table of 4-byte unsigned integers in network order. The ith
-+ value is the modification time (mtime) of the ith object in the
-+ corresponding pack by lexicographic (index) order. The mtimes
-+ count standard epoch seconds.
++ - A table of 4-byte unsigned integers. The ith value is the
++ modification time (mtime) of the ith object in the corresponding
++ pack by lexicographic (index) order. The mtimes count standard
++ epoch seconds.
+
+ - A trailer, containing a checksum of the corresponding packfile,
+ and a checksum of all of the above (each having length according
+ to the specified hash function).
-+
-+All 4-byte numbers are in network order.
+
== multi-pack-index (MIDX) files have the following format:
2: cdb21236e1 = 3: 67c4e7209d pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles'
3: 1d775f9850 = 4: fc86506881 chunk-format.h: extract oid_version()
4: 6172861bd9 = 5: 788d1f96f2 pack-mtimes: support writing pack .mtimes files
5: 5f9a9a5b7b = 6: 2a6cfb00bf t/helper: add 'pack-mtimes' test-tool
6: b8a38fe2e4 = 7: edb6fcd5ec builtin/pack-objects.c: return from create_object_entry()
7: 94fe03cc65 = 8: e3185741f2 builtin/pack-objects.c: --cruft without expiration
8: da7273f41f = 9: 1cf00d462c reachable: add options to add_unseen_recent_objects_to_traversal
9: 58fecd1747 = 10: d66be44d9a reachable: report precise timestamps from objects in cruft packs
10: 1740b8ef01 = 11: 1434e37623 builtin/pack-objects.c: --cruft with expiration
11: 5992a72cbf ! 12: 0d3555d595 builtin/repack.c: support generating a cruft pack
@@ t/t5329-pack-objects-cruft.sh: test_expect_success 'expired objects are pruned'
+ git repack &&
+
+ tip="$(git rev-parse cruft)" &&
-+ path="$objdir/$(test_oid_to_path "$(git rev-parse cruft)")" &&
++ path="$objdir/$(test_oid_to_path "$tip")" &&
+ test-tool chmtime --get +1000 "$path" >expect &&
+
+ git checkout main &&
12: 1b241f8f91 = 13: 4b721d3ee9 builtin/repack.c: allow configuring cruft pack generation
13: ffae78852c = 14: f9e3ab56b1 builtin/repack.c: use named flags for existing_packs
14: 0743e373ba ! 15: e9f46e7b5e builtin/repack.c: add cruft packs to MIDX during geometric repack
@@ builtin/repack.c
static int pack_everything;
static int delta_base_offset = 1;
@@ builtin/repack.c: static void collect_pack_filenames(struct string_list *fname_nonkept_list,
+ fname = xmemdupz(e->d_name, len);
+
if ((extra_keep->nr > 0 && i < extra_keep->nr) ||
- (file_exists(mkpath("%s/%s.keep", packdir, fname))))
+- (file_exists(mkpath("%s/%s.keep", packdir, fname))))
++ (file_exists(mkpath("%s/%s.keep", packdir, fname)))) {
string_list_append_nodup(fname_kept_list, fname);
- else
- string_list_append_nodup(fname_nonkept_list, fname);
-+ else {
-+ struct string_list_item *item = string_list_append_nodup(fname_nonkept_list, fname);
++ } else {
++ struct string_list_item *item;
++ item = string_list_append_nodup(fname_nonkept_list,
++ fname);
+ if (file_exists(mkpath("%s/%s.mtimes", packdir, fname)))
+ item->util = (void*)(uintptr_t)CRUFT_PACK;
+ }
15: 9f7e0acac6 = 16: 43c14eec07 builtin/gc.c: conditionally avoid pruning objects via loose
16: 07fa9d4b47 = 17: 1e313b89e8 sha1-file.c: don't freshen cruft packs
--
2.36.1.94.gb0d54bedca
