Re: Anomaly with the new code - Re: git-svn performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hin-Tak Leung <htl10@xxxxxxxxxxxxxxxxxxxxx> wrote:
> I keep tabs of a particular svn repository over many years
> and run "git svn fetch --all" every few days. So that's the old clone.
> Since this discussion started, I made a new one with git 2.1.0 patched
> with the first two patches below, a couple of weeks ago. And I ran
> 'git svn fetch --all' on both every few days since.
> 
> I have added a few more patches, so the whole list is the 6
> below against 2.1.0. The latest fetch is really strange - the fetch against
> the new clone took almost twice as long and uses almost twice
> as much memory, vs against the old. 17 min, 800 MB vs 10 min 400MB.
> Details below. Maybe this is a performance issue about how the clones
> were made?

Memory usage seems to grow with the amount of revisions fetched,
see below.  And higher memory means slower fork() on Linux systems.

> 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch    
> 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch   
> 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch        
> 0004-git-svn-cache-only-mergeinfo-revisions.patch                 

> 0006-git-svn-clear-global-SVN-pool-between-get_log-invoca.patch   

0006 is insufficient and incompatible with older SVN.
I pushed "git-svn: reload RA every log-window-size"
(commit dfa72fdb96befbd790f623bb2909a347176753c2) instead
which saves much more memory:

http://mid.gmane.org/20141024225352.GB31716@xxxxxxxxxxxxx

But there still seems to be some slow growth with many revisions
which is not mergeinfo-related.

> 0007-git-svn-remove-mergeinfo-rev-caching.patch 

I think it is also safe to remove the _rev_list memoization since
it uses a lot of memory.  The remaining caches should be tiny
(but useful, I think).
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]