Hi all,
[spoiler alert: I've figured out the config option causing the problem
while writing this long mail, so you might jump straight to the SOLUTION
section at the bottom of this mail.]
at my day job, I work on a git repo (sadly non-public, proprietary) with
these stats:
- master has about 150000 commits, the last release branch I've also benchmarked above has 144000 commits
- the history dates back to 2001
- .git/ is about 1.8 GB
So it's quite big but not unusually big when compared to linux or other
free software projects.
The typical git commands I use (status, fetch, pull, commit, push,
rebase, ...) are all quick. However, I use the git porcelain Magit [1]
which invokes several plumbing commands in order to present to the user
an always up-to-date extended status buffer of the currently checked out
branch showing the current branch. Some of those plumbing commands are
extremely slow for no obvious reasons. The most outstanding command I
could pinpoint is this:
--8<---------------cut here---------------start------------->8---
❯ time git show --no-patch --format="%h %s" "master^{commit}" --
6192a0cfdc6 Merge remote-tracking branch 'origin/SHD_ECORO_3_9_7'
________________________________________________________
Executed in 13.21 secs fish external
usr time 12.99 secs 462.00 micros 12.99 secs
sys time 0.17 secs 119.00 micros 0.17 secs
--8<---------------cut here---------------end--------------->8---
The interesting thing is that I have this problem only with the master
branch. When I run it for the last release branch, I get these times:
--8<---------------cut here---------------start------------->8---
❯ time git show --no-patch --format="%h %s" "SHD_ECORO_3_9_7^{commit}" --
994334fc9fb ECOJ-33833 HTML-Formbrief: Bestellungs-Anhänge im KV-Kontext
________________________________________________________
Executed in 22.68 millis fish external
usr time 7.71 millis 761.00 micros 6.95 millis
sys time 10.47 millis 194.00 micros 10.28 millis
--8<---------------cut here---------------end--------------->8---
So you see, it's almost a factor of 1000 difference! How can that be?
The split between master and the SHD_ECORO_3_X_X series of branches has
happened almost 2 years ago and master is way ahead of those.
--8<---------------cut here---------------start------------->8---
❯ git log --oneline master...origin/SHD_ECORO_3_9_7 | wc -l
5013
--8<---------------cut here---------------end--------------->8---
But there are around 9 merges from the last release branch into master
daily.
--8<---------------cut here---------------start------------->8---
❯ git log --merges --oneline --since 6months | wc -l
1611
--8<---------------cut here---------------end--------------->8---
>From my memory, the issue hasn't popped up out of sudden but has gotten
worse slowly over time. I have the impression that the worsening
increased pace over the last few month which might be the result of our
workflow. Before, I've been the merge guy doing two "merge waves" from
the last supported release branch upwards into master once or twice a
day (usually release-branch -> next-release-branch -> master). Since
about 3 month, we've switched to a workflow where every developer does
merge upwards herself just after committing/pushing to some lesser
branch than master simply because branches have diverged so much that
you'd need to be an expert in everything in order to be able to resolve
conflicts sensibly.
I should mention that I haven't seen this issue with any other repo I
have. But that's also the biggest one I use. The Emacs repository I
also work on is comparable in the number of commits but with much less
merges.
At last, here's the git bugreport sysinfo section on that machine and
repository.
--8<---------------cut here---------------start------------->8---
[System Info]
git version:
git version 2.36.1
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 5.18.1-zen1-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Mon, 30 May 2022 17:53:16 +0000 x86_64
compiler info: gnuc: 11.2
libc info: glibc: 2.35
$SHELL (typically, interactive shell): /usr/bin/fish
[Enabled Hooks]
--8<---------------cut here---------------end--------------->8---
SOLUTION
========
While writing this long mail, I've figured out that the performance
penalty is caused by my setting of diff.renameLimit = 10000. If I
comment that option in my ~/.gitconfig, the above command finishes in
150 millis instead of 13 seconds:
--8<---------------cut here---------------start------------->8---
❯ time git show --no-patch --format="%h %s" "master^{commit}" --
6192a0cfdc6 Merge remote-tracking branch 'origin/SHD_ECORO_3_9_7'
________________________________________________________
Executed in 147.99 millis fish external
usr time 114.52 millis 713.00 micros 113.81 millis
sys time 34.78 millis 193.00 micros 34.59 millis
--8<---------------cut here---------------end--------------->8---
But there's still the question why diff.renameLimit has an influence
here when --no-patch is provided so no diff should be generated.
Bye,
Tassilo
[1] https://magit.vc/
