Re: Git push performance problems with ~100K refs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Mar 30, 2012 at 05:32:08AM -0400, Jeff King wrote:

> But that's just a suspicion. I don't have time tonight to work out a
> test case. Is it possible for you to run something like:
> 
>   # make a new commit on top of HEAD, but not yet referenced
>   sha1=`git commit-tree HEAD^{tree} -p HEAD </dev/null`
> 
>   # now do the same "connected" test that receive-pack would do
>   git rev-list --objects $sha1 --not --all
> 
> That should replicate the slow behavior you are seeing. If that works,
> try running the latter command under "perf"; my guess is that you will
> see commit_list_insert_by_date as a hot-spot.
> 
> Even doing this simple test on a moderate repository (my git.git has
> ~1100 refs), commit_list_insert_by_date accounts for 10% of the CPU
> according to perf.

Actually, I did have time for a simple test. Doing:

  git rev-list HEAD |
  while read sha1; do
    echo $sha1 refs/heads/$sha1
  done >>packed-refs
  git pack-refs

in git.git slows down the test above considerably, and perf reports 90%
of the time spent in commit_list_insert_by_date. So I think that is
indeed the problem.

At one point, I looked at replacing the commit_list implementation with
a heap-based priority queue, but unfortunately many parts of the code
depend on the list-like nature and would need to be rewritten. We might
be able to hack around it by at least adding all of the initial items to
an unordered list, then sorting it into its final form.

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]