Re: git gc --aggressive led to about 40 times slower "git log --raw"

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Feb 22, 2014 at 4:14 PM, Duy Nguyen <pclouds@xxxxxxxxx> wrote:
> On Sat, Feb 22, 2014 at 3:53 PM, David Kastrup <dak@xxxxxxx> wrote:
>> David Kastrup <dak@xxxxxxx> writes:
>>
>>> Duy Nguyen <pclouds@xxxxxxxxx> writes:
>>>
>>>> OK with git://git.savannah.gnu.org/emacs.git we have
>>>>
>>>>  - a 209MB pack with --aggressive
>>>>  - 1.3GB with --depth=50
>>>>  - 1.3GB with --window=4000 --depth=32
>>>>  - 1.3GB with --depth=20
>>>>  - 821MB with --depth=250 for commits --before=2.years.ago, --depth=50
>>>> for the rest
...
>>>>
>>>> I'm not really happy with --depth=250 producing 209MB while
>>>> --depth=250 --before=2.year.ago a 800MB pack. It looks wrong (or maybe
>>>> I did something wrong)
....
>> Another thing: did you really use --depth=250 here or did you use
>> --aggressive?  It may be that the latter also sets other options?
>
> I can't use --aggressive because I need to feed revisions directly to
> pack-objects. --aggressive also sets --window=250. Thanks for
> checking. My machine will have another workout session.

And 800MB is reduced to 177MB, containing history older than 2 years.
The final pack is 199MB, within the size range of current --aggressive
and should be reasonably fast on most operations. Again blame could
still hit long delta chains but I think we should just unpack some
trees/blobs when we hit long delta chains.

I think we should update --aggressive to do it this way. So

 - gc.aggressiveDepth defaults to 50 (or 20?), this is used for recent history
 - gc.aggressiveDeepDepth defaults to 250 (or smaller??), used for
ancient history
 - gc.aggressiveDeepOption is rev-list a rev-list option to define
"ancient history", default to --before=2.years.ago. This option could
be specified multiple times.

Both packing phases use the same gc.aggressiveWindow. We could add
gc.aggressiveDeepWindow too.

GSoC project?
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]