Re: Weird growth in packfile during initial push

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Nicolas Pitre <nico@xxxxxxx> writes:

>> $ git push origin master:master
>> Initialized empty Git repository in /var/gitroot/exp/gentoo-x86.git/
>> Counting objects: 4969800, done.
>> Delta compression using up to 8 threads.
>> Compressing objects: 100% (1217809/1217809), done.
>> Writing objects: 100% (4969800/4969800), 810.56 MiB | 21608 KiB/s, done.
>> Total 4969800 (delta 3735812), reused 4969800 (delta 3735812)
>
> Here we know for sure that all objects were directly reused, so no 
> attempt at recompressing them was done.  The only thing that 
> pack-objects might do in this case in addition to directly streaming the 
> existing pack is to convert delta object headers from OFS_DELTA to 
> REF_DELTA.
>
>> $ ls -la /var/gitroot/exp/gentoo-x86.git/objects/pack
>> total 966876
>> drwxr-xr-x 2 git git      4096 Apr 14 08:43 .
>> drwxr-xr-x 4 git git      4096 Apr 14 08:35 ..
>> -r--r--r-- 1 git git 139155472 Apr 14 08:43 pack-f805bb448f864becfeac9c7f8a8ac2ef90c26787.idx
>> -r--r--r-- 1 git git 849936308 Apr 14 08:43 pack-f805bb448f864becfeac9c7f8a8ac2ef90c26787.pack
>
> Let's see if my theory stands:
>
> 	849936308 - 786336481 = 63599827
> 	63599827 / 3735812 = 17.02
>
> Hence an average difference of 17 bytes per delta.  Given that REF_DELTA 
> objects have a 20-byte SHA1 base reference which is replaced with a 
> variable length encoding of a pack offset in the OFS_DELTA case, we're 
> talking about 2.98 bytes for that offset encoding which feels about 
> right.
>
> [...]
>
> And the code matches this theory as well.  Can you try this patch if you 
> have a chance?

Is there any progress on this?

I think you did a veryclear analysis.  8% size reduction is not only
unignorable but use of delta offset should also help runtime efficiency,
right?



--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]