Re: [PATCH 7/9] fetch: fetch submodules in parallel

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Aug 28, 2015 at 11:35 AM, Jeff King <peff@xxxxxxxx> wrote:
> On Fri, Aug 28, 2015 at 11:27:04AM -0700, Junio C Hamano wrote:
>
>> > But for commands that show progress like "git clone", "git checkout",
>> > and "git fetch", it does not work well at all.  They provide output
>> > that updates itself by putting a carriage return at the end of each
>> > chunk of output, like this:
>> >
>> >  remote: Finding sources:  11% (18/155)           \r
>> >  remote: Finding sources:  12% (19/155)           \r
>> >
>> > With multiple commands producing such output, they will overwrite each
>> > other's lines, producing a mixture that is confusing and unuseful.
>>
>> That example also illustrates why it is not a useful to buffer all
>> of these lines and showing them once.
>
> I think Jonathan's point is that you could pick _one_ active child to
> show without buffering, while simultaneously buffering everybody else's
> output. When that finishes, pick a new active child, show its buffer,
> and then start showing its output in realtime. And so on.

or better yet, pick that child with the most progress (i.e. flush all finished
children and then pick the next active child), that would approximate
the progress in the output best, as it would reduce the hidden
buffered progress.

>
> So to an observer, it would look like a serial operation, but subsequent
> operations after the first would magically go much faster (because
> they'd been working and buffering in the background).
>
> And that doesn't require any additional IPC magic (though I am not sure
> how we get progress in the first place if the child stderr is a
> pipe...).

Moving the contents from the pipe to a strbuf buffer which we can grow
indefinitely
(way larger than pipe limits, but the output of a git fetch should be
small enough for that).



>
> -Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]