Re: git-import.sh using git-fast-import

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Jeff King <peff@xxxxxxxx> wrote:
> Just for giggles, here's the timing of the "core" and "perl" versions,
> both with a "git repack -a -d -f" afterwards.
> 
> core:
> real    0m3.353s
> user    0m2.108s
> sys     0m0.580s
> 
> perl:
> real    0m3.138s
> user    0m2.704s
> sys     0m0.236s
> 
> So the perl version actually wins here, though with two caveats:
>   - the core version spends more sys time. Presumably this is because it
>     actually gives you a working index, whereas the fast-import version
>     does not.

It may also have to do with the large amount of data going over a
pipe, as well as the fact that we have to write the data into the
packfile then go back and reread all of it to compute the SHA-1 of
the entire thing.  That's a lot of IO, all going through your poor
little kernel.  ;-)

>   - the perl version actually consumes _more_ CPU time, but finishes
>     more quickly. Yes, this is an SMP box. :)

Yea, so like the parallel pack-objects experiment that Nico had
done recently we decreased wall-clock time at the expense of using a
larger amount of the system resources.  That is bad as we use more
CPU time than we saved in wallclock time.  Usually a bad tradeoff,
unless you have a realtime requirement you have to meet.


Thanks for running this, it was interesting to see.  There are a
lot of ways that fast-import could be improved to be faster-import
(heh) but we haven't tried to do anything as it has typically been
fast enough.  Some of the ideas have been discussed on list, but
they just haven't been implemented.  One problem is fast-import
is O(n^2 + n log n) in updating its internal tree state.  Not very
fast on trees with a lot of entries in them.  At least fast-import
doesn't use a flat structure like the index.

-- 
Shawn.
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux