Re: Git is not scalable with too many refs/*

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, 26 Sep 2011 12:07:52 -0600, Martin Fick wrote:
-- snip --
Ahh, I think I have some more clues.  So while this change
does not speed things up for me normally, I found a case
where it does!  I  set my .git/config to have

  [core]
        compression = 0

and ran git-gc on my repo.  Now, with a modern git with this
optimization in it (1.7.6, 1.7.7.rc0...), 'git branch' is
almost instantaneous (.05s)!  But, if I revert c774aa it
takes > ~15s.

I don't understand this. I don't see why core.compression should have anything to do with refs ...

So, it appears that this optimization is foiled by
compression?  In the case when this optimization helps, it
save about 15s, when it hurts (with compression), it seems
to cost > 3mins.  I am not sure this optimization is worth
it?  Would there be someway for it to adjust to the repo
conditions?

Well, in the case I tried it was 1.2s vs 40s. It would seem that you have managed to find some corner case. It doesn't seem right to punish everyone who has large numbers of refs by making their commands take orders of magnitude longer to save one person 3m. Much better to find, understand and fix the actual cause.

I really can't see what effect core.compression can have on loading the ref_list. Certainly the sort doesn't load anything from the object database. It would be really good to profile and find out what is taking all the time - I am assuming that the CPU is at 100% for the 3+ minutes?

Random thought. What happens to the with compression case if you leave the commit in, but add a sleep(15) to the end of sort_refs_list?

--
Julian

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]