Re: Parallel checkout (Was Re: 0 bot for Git)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Apr 15, 2016 at 4:51 PM, Duy Nguyen <pclouds@xxxxxxxxx> wrote:
> Numbers are encouraging though. On linux-2.6 repo running on linux and
> ext4 filesystem, checkout_paths() would dominate "git checkout :/".
> Unmodified git takes about 31s.
>
>
> 16:26:00.114029 builtin/checkout.c:1299 performance: 31.184973659 s: checkout_paths
> 16:26:00.114225 trace.c:420             performance: 31.256412935 s: git command: 'git' 'checkout' '.'
>
> When doing write_entry() on 8 processes, it takes 22s (shortened by ~30%)

I continued to develop it into a series. This same laptop now reduces
checkout time closer to 50% on linux-2.6. However my other laptop
gives me the opposite result, parallel checkout takes longer time to
complete. I suspect that only with fast enough disks that CPU may
become temporary bottleneck. This is where parallel checkout shines
because it spreads the load out and quickly moves the bottleneck back
to I/O (after a while I/O queues should be fully populated again). On
systems with slower disks like mine, I/O is always the bottleneck and
spreading I/O over many processes just makes it worse (probably
confuse I/O scheduler more).

Since it's not doing anything for _me_, I'm dropping this. Anybody
interested can check it out and maybe try it from parallel-checkout
branch [1]. It probably can build on windows (epoll is gone). And it
probably help improve performance when smudge filter is used (because
that can potentially add more load to cpu). More notes in commit
8fe9b5c (entry.c: parallel checkout support - 2016-04-18)

[1] https://github.com/pclouds/git/commits/parallel-checkout
-- 
Duy
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]