Re: Poor performance of git describe in big repos

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Alex Bennée wrote:
> 15:50 ajb@sloy/x86_64 [work.git] >time git log --pretty=oneline | wc -l
> 24648
>
> real    0m0.434s
> user    0m0.388s
> sys     0m0.112s
>
> Although it doesn't take too long to walk the whole mainline history
> (obviously ignoring all the other branches).

Damn, non-starter.  linux.git has 361k+ commits in mainline history.

Nit: use git rev-list --count HEAD next time.

> 15:52 ajb@sloy/x86_64 [work.git] >git count-objects -v -H
> count: 581
> size: 5.09 MiB
> in-pack: 399307
> packs: 1
> size-pack: 1.49 GiB
> prune-packable: 0
> garbage: 0
> size-garbage: 0 bytes

linux.git has 2.9m+ in-pack.  The pack-size is much lower at about
800+ MiB, but I don't think 1.49 GiB is a problem in itself.  Looking
forward to your big-files report to see why it's so big.

> It is a pick repo. The gc --aggressive nearly took out my machine keeping
> around 4gb resident for most of the half hour and using nearly 8gb of VM.
>
> Of course most of the history is not needed for day to day stuff. Maybe
> if I split the pack files up it wouldn't be quite such a strain to work
> through them?

Really out of my depth here, sorry.  Let's see what Duy (or the
others) have to say.

>> 2. You have have huge (binary) files checked into your repository.  Do
>> you?  If so, why isn't the code in streaming.c kicking in?
>
> We do have some binary blobs in the repository (mainly DSP and FPGA images)
> although not a huge number:
>
> 15:58 ajb@sloy/x86_64 [work.git] >time git log --pretty=oneline -- xxx
> xxx/xxxxxx/*.out ./xxx/xxx/*.out ./xxx/xxxxxxx/*.out | wc -l
> 234
>
> real    0m0.590s
> user    0m0.552s
> sys     0m0.040s

log is streaming, and is not a good measure: it doesn't even walk the
entire commit graph.  How big are these files?

> How can I tell if streaming is kicking in or now?

I use callgrind (and kcachegrind to visualize).  Can you post
callgrind output?  It will be helpful in figuring out where exactly
git is spending time.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]