I had a user report some poor behavior of 'git gc --aggressive' on a
certain repo (which I sadly cannot share). Turns out that on this
repo, this operation takes about 60% longer and produces a pack
roughly twice the expected size.
Naturally, bisecting takes a while but it points to this commit:
0aca34e8269514ebb67676e0470a314615494ae8 is the first bad commit
commit 0aca34e8269514ebb67676e0470a314615494ae8
Author: Nguyễn Thái Ngọc Duy <pclouds@xxxxxxxxx>
Date: Sat Apr 14 17:35:11 2018 +0200
pack-objects: shrink delta_size field in struct object_entry
Allowing a delta size of 64 bits is crazy. Shrink this field down to
20 bits with one overflow bit.
If we find an existing delta larger than 1MB, we do not cache
delta_size at all and will get the value from oe_size(), potentially
from disk if it's larger than 4GB.
Note, since DELTA_SIZE() is used in try_delta() code, it must be
thread-safe. Luckily oe_size() does guarantee this so we it is
thread-safe.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@xxxxxxxxx>
Signed-off-by: Junio C Hamano <gitster@xxxxxxxxx>
To put some numbers behind this, I got on a very beefy box (40 cores,
160GB RAM) and ran some comparisons:
Version Pack (MB) MaxRSS(kB) Time (s)
------- --------- ---------- --------
2.17.0 5498 43513628 2494.85
2.18.0 10531 40449596 4168.94
fix-v1 5509 42509784 2480.74
fiv-v2 5509 41644104 2468.25
where fix-v1 and fix-v2 are different patches on top of git-2.18.0
that I'll follow up to this email with. The patches are just meant as
discussion starters. I'll add signoffs and proper commit messages if
folks actually want those fixes, but I suspect they're just starting
points for discussion.
