Re: Git push performance problems with ~100K refs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




Jeff King <peff@xxxxxxxx> wrote:
>
>Actually, I did have time for a simple test. Doing:
>
>  git rev-list HEAD |
>  while read sha1; do
>    echo $sha1 refs/heads/$sha1
>  done >>packed-refs
>  git pack-refs
>
>in git.git slows down the test above considerably, and perf reports 90%
>of the time spent in commit_list_insert_by_date. So I think that is
>indeed the problem.
>
>At one point, I looked at replacing the commit_list implementation with
>a heap-based priority queue, but unfortunately many parts of the code
>depend on the list-like nature and would need to be rewritten. We might
>be able to hack around it by at least adding all of the initial items
>to
>an unordered list, then sorting it into its final form.

Thanks Peff for the explanation.  Jgit actually has the exact same problem, it slows down the pushing side.  Fortunately, in jgit it is well isolated and can easily be remedied by both the solutions you mention, and both work to speed it (jgit) up drastically!  I wonder if libgit2 suffers from the same problem?

This might be one of the last pieces to git ref scalability left?  Does this affect many other use cases, fetches?

-Martin


Employee of Qualcomm Innovation Center,Inc. which is a member of Code Aurora Forum
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]