On Thu, Jan 10, 2008 at 08:39:07PM +0000, Nicolas Pitre wrote:
> On Thu, 10 Jan 2008, Pierre Habouzit wrote:
> diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
> index a39cb82..252b03e 100644
> --- a/builtin-pack-objects.c
> +++ b/builtin-pack-objects.c
> @@ -433,7 +433,10 @@ static unsigned long write_object(struct sha1file *f,
> }
> /* compress the data to store and put compressed length in datalen */
> memset(&stream, 0, sizeof(stream));
> - deflateInit(&stream, pack_compression_level);
> + if (obj_type == OBJ_REF_DELTA || obj_type == OBJ_OFS_DELTA)
> + deflateInit(&stream, 0);
> + else
> + deflateInit(&stream, pack_compression_level);
> maxsize = deflateBound(&stream, size);
> out = xmalloc(maxsize);
> /* Compress it */
>
> You then only need to run 'git repack -a -f -d' with and without the
> above patch.
Using as a PoC a test that is if (size <= 512) instead, I get:
vanilla git:
$ du -k .git/**/*.pack
180808 .git/objects/pack/pack-7bc9f383c92cbffe366da2d2a62b67bb33a53365.pack
$ repeat 5 time git blame MAINTAINERS >|/dev/null
git blame MAINTAINERS >| /dev/null 7,34s user 0,09s system 99% cpu 7,433 total
git blame MAINTAINERS >| /dev/null 7,31s user 0,16s system 100% cpu 7,475 total
git blame MAINTAINERS >| /dev/null 7,35s user 0,08s system 100% cpu 7,431 total
git blame MAINTAINERS >| /dev/null 7,30s user 0,18s system 99% cpu 7,482 total
git blame MAINTAINERS >| /dev/null 7,33s user 0,16s system 99% cpu 7,492 total
With a compression disabled for sizes <= 512:
$ du -k .git/**/*.pack
188840.git/objects/pack/pack-7bc9f383c92cbffe366da2d2a62b67bb33a53365.pack
$ repeat 5 time git blame MAINTAINERS >|/dev/null
git blame MAINTAINERS >| /dev/null 7,06s user 0,09s system 100% cpu 7,150 total
git blame MAINTAINERS >| /dev/null 7,08s user 0,13s system 99% cpu 7,209 total
git blame MAINTAINERS >| /dev/null 7,07s user 0,08s system 99% cpu 7,168 total
git blame MAINTAINERS >| /dev/null 7,02s user 0,15s system 99% cpu 7,177 total
git blame MAINTAINERS >| /dev/null 7,07s user 0,13s system 99% cpu 7,243 total
Okay, the size doesn't even budge, it's not even near being fun. Though
we gain 3% of wall clock time
Let's try with a limit of 1024 then !
$ du -k .git/**/*.pack
201725 .git/objects/pack/pack-7bc9f383c92cbffe366da2d2a62b67bb33a53365.pack
$ repeat 5 time git blame MAINTAINERS >|/dev/null
git blame MAINTAINERS >| /dev/null 6,93s user 0,16s system 77% cpu 9,109 total
git blame MAINTAINERS >| /dev/null 6,88s user 0,08s system 99% cpu 6,965 total
git blame MAINTAINERS >| /dev/null 6,84s user 0,10s system 99% cpu 6,952 total
git blame MAINTAINERS >| /dev/null 6,86s user 0,12s system 99% cpu 6,983 total
git blame MAINTAINERS >| /dev/null 6,81s user 0,18s system 99% cpu 6,994 total
Okay, the packs grows 10%, and the blame takes 6% less time.
Okay the numbers are still not that impressive, but my patch doesn't
touches _only_ deltas, but also log comments I said, so I've redone my
tests with git log and *TADAAAA*:
vanilla git:
repeat 5 time git log >|/dev/null
git log >| /dev/null 2,54s user 0,12s system 99% cpu 2,660 total
git log >| /dev/null 2,52s user 0,12s system 99% cpu 2,653 total
git log >| /dev/null 2,57s user 0,07s system 99% cpu 2,637 total
git log >| /dev/null 2,56s user 0,09s system 99% cpu 2,659 total
git log >| /dev/null 2,54s user 0,10s system 99% cpu 2,660 total
with the 512 octets limit:
$ repeat 5 time git log >|/dev/null
git log >| /dev/null 2,10s user 0,10s system 99% cpu 2,193 total
git log >| /dev/null 2,08s user 0,10s system 99% cpu 2,189 total
git log >| /dev/null 2,06s user 0,11s system 100% cpu 2,162 total
git log >| /dev/null 2,04s user 0,13s system 100% cpu 2,172 total
git log >| /dev/null 2,06s user 0,13s system 99% cpu 2,198 total
That's already a 20% time reduction.
with the 1024 octets limits:
$ repeat 5 time git log >|/dev/null
git log >| /dev/null 1,39s user 0,12s system 99% cpu 1,512 total
git log >| /dev/null 1,38s user 0,12s system 100% cpu 1,498 total
git log >| /dev/null 1,41s user 0,10s system 99% cpu 1,514 total
git log >| /dev/null 1,41s user 0,10s system 100% cpu 1,506 total
git log >| /dev/null 1,40s user 0,10s system 100% cpu 1,504 total
Yes that's 43% time reduction !
As a side note, repacking with the 1024 octets limits takes 4:06 here,
and 4:26 without the limit at all, which is 8% less time. I know it
doesn't matters a lot as repack is a once time operation, but still, it
would speed up git gc --auto which is not something to neglect
completely.
I say it's worth investigating a _lot_, and the patch is that complicated:
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index a39cb82..f454929 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -433,7 +433,7 @@ static unsigned long write_object(struct sha1file *f,
}
/* compress the data to store and put compressed length in datalen */
memset(&stream, 0, sizeof(stream));
- deflateInit(&stream, pack_compression_level);
+ deflateInit(&stream, size > 1024 ? pack_compression_level : 0);
maxsize = deflateBound(&stream, size);
out = xmalloc(maxsize);
/* Compress it */
--
·O· Pierre Habouzit
··O madcoder@xxxxxxxxxx
OOO http://www.madism.org
Attachment:
pgpLa3MgVQIml.pgp
Description: PGP signature
