Partitioned packs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I've been running some experiments, as hinted earlier by the
discussion about just how much git-index-pack sucks (which, really,
isn't much since the gaping memleak is gone now).

These experiments include trying to see if there's a noticeable
performance improvement by splitting out objects of different types
into different packs. So far, it definitely seems to make a
difference, though not the one I was initially expecting. For all of
these tests, I did 'sysctl -w vm.drop_caches=3' before running, to
effectively simulate a cold-cache run.

Single 3.1GB pack file containing all commits, blobs, and trees
First run (cold cache):
git-rev-list --all > /dev/null  5.52s user 0.32s system 45% cpu 12.872 total
git-blame -- kdelibs/kdeui/kmenubar.cpp  0.00s user 0.01s system 0%
cpu 40.218s total
git-archive --format=tar HEAD -- kdelibs >> /dev/null  0.48s user
0.10s system 5% cpu 10.143 total

Subsequent runs (warm cache):
git-rev-list --all > /dev/null  5.19s user 0.48s system 98% cpu 5.750 total
git-blame -- kdelibs/kdeui/kmenubar.cpp  0.00s user 0.00s system 0%
cpu 11.960 total
git-archive --format=tar HEAD -- kdelibs >> /dev/null  0.43s user
0.04s system 100% cpu 0.472 total


Single pack for commit objects and another pack for the rest
First run (cold cache):
git-rev-list --all > /dev/null  5.84s user 0.34s system 31% cpu 19.427 total
git-blame -- kdelibs/kdeui/kmenubar.cpp  0.00s user 0.00s system 0%
cpu 9:42.74 total
git-archive --format=tar HEAD -- kdelibs >> /dev/null  0.50s user
0.26s system 0% cpu 1:35.44 total

Subsequent runs (warm cache):
git-rev-list --all > /dev/null  5.94s user 0.26s system 99% cpu 6.204 total
git-blame -- kdelibs/kdeui/kmenubar.cpp  0.00s user 0.00s system 0%
cpu 12.394 total
git-archive --format=tar HEAD -- kdelibs >> /dev/null  0.41s user
0.07s system 98% cpu 0.486 total

Fully-partitioned separate packs for commit, tree, and blob objects
First run (cold cache):
git-rev-list --all > /dev/null  6.24s user 0.32s system 25% cpu 25.689 total
git-blame -- kdelibs/kdeui/kmenubar.cpp  0.00s user 0.00s system 0%
cpu 1:08.76 total
git-archive --format=tar HEAD -- kdelibs >> /dev/null  0.38s user
0.30s system 0% cpu 1:35.89 total

Subsequent runs (warm cache):
git-rev-list --all > /dev/null  6.28s user 0.24s system 99% cpu 6.527 total
git-blame -- kdelibs/kdeui/kmenubar.cpp  0.00s user 0.00s system 0%
cpu 13.895 total
git-archive --format=tar HEAD -- kdelibs >> /dev/null  0.42s user
0.06s system 99% cpu 0.476 total

I packed all of these using --delta-base-offset, with a window of 100
and a depth of 10.

-clee
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]