Re: local clone performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 7/27/06, Petr Baudis <pasky@xxxxxxx> wrote:
Dear diary, on Thu, Jul 27, 2006 at 01:38:00AM CEST, I got a letter
where Jon Smirl <jonsmirl@xxxxxxxxx> said that...
> Comparing git-clone to cg-clone
>
> [jonsmirl@jonsmirl apps]$ time git-clone git foo
> ......
> real    0m5.755s
> user    0m4.548s
> sys     0m0.512s
>
> [jonsmirl@jonsmirl apps]$ time cg-clone git foo
> ....
> real    0m18.970s
> user    0m10.737s
> sys     0m1.392s
>
> Why does it take cg clone so long to set up the missing tags?
> git-clone is over 3x faster.

  How many tags do you have?

I am cloning the git source tree. It looks to have about 40 tags.

  It's true that cg-clone does not scale very well with big amounts of
tags, because it won't fetch tagged objects you wouldn't get otherwise
(so if you tagged some random huge blob or a history line which is not
part of any branch you have, cg-clone won't grab it all). This
unfortunately costs us one special fetch invocation per tag; I will look
into hacking git-local-fetch to accept multiple commit ids at once, that
should speed it up considerably.

  That said, with cg-clone -a I will be able to cut that difference
entirely and blindly take everything from the other repository.

  Could you please try the patch below and check if it has at least
any measurable impact on the performance at all? Thanks.

New timings
real    0m11.259s
user    0m9.977s
sys     0m1.280s

That's a 40% improvement.

diff --git a/cg-fetch b/cg-fetch
index a6e6959..23e0e77 100755
--- a/cg-fetch
+++ b/cg-fetch
@@ -54,7 +54,6 @@ _git_wc_unneeded=1

 fetch_progress()
 {
-       [ $verbose -ge 2 ] && exec cat
        if [ -t 1 ]; then
                exec "${COGITO_LIB}"cg-Xfetchprogress "$_git_objects"
        else
@@ -166,7 +165,11 @@ fetch_http()
 {
        whead=
        [ "$3" ] && whead="-w $3"
-       (git-http-fetch -a -v $whead $recovery "$1" "$2/" 2>&1 /dev/null) | fetch_progress
+       if [ $verbose -ge 2 ]; then
+               git-http-fetch -a -v $whead $recovery "$1" "$2/"
+       else
+               (git-http-fetch -a -v $whead $recovery "$1" "$2/" 2>&1 /dev/null) | fetch_progress
+       fi
        return ${PIPESTATUS[0]}
 }

@@ -197,7 +200,11 @@ fetch_local()
 {
        whead=
        [ "$3" ] && whead="-w $3"
-       (git-local-fetch -a -l -v $whead $recovery "$1" "$2" 2>&1 /dev/null) | fetch_progress
+       if [ $verbose -ge 2 ]; then
+               git-local-fetch -a -l -v $whead $recovery "$1" "$2"
+       else
+               (git-local-fetch -a -l -v $whead $recovery "$1" "$2" 2>&1 /dev/null) | fetch_progress
+       fi
        return ${PIPESTATUS[0]}
 }

@@ -230,6 +237,7 @@ fetch_tags()
                        # if so, fetch the tag -- which should be
                        # a cheap operation -- to complete the chain.
                        echo -n "Missing tag ${tagname#tags/}... "
+                       local verbose=2
                        if $fetch "$tagname" "$uri" "$tagname" 2>/dev/null >&2; then
                                echo "retrieved"
                        else


--
                                Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Snow falling on Perl. White noise covering line noise.
Hides all the bugs too. -- J. Putnam



--
Jon Smirl
jonsmirl@xxxxxxxxx
-
: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]