This is a new approach to making git submodules faster without using threads. We'll directly run commands in parallel without an intermediate thread pool. patch 1 is fixing an error, which was hard to come by in a parallel world, but is not much of a problem in a single-childed world. The second patch works fine, (I'd love to hear feedback on the percieved change of output) though it has still two edges I want to tackle: * The fcntl call to make the pipes nonblocking is not checked for errors. * We also need to communicate back if one process cannot spawn or such, i.e. the reporting for all subprocesses combined needs to added. I think about adding another callback which is called to finish up a child process. Earlier attempts used a threading pool, which made the conversion very easy as you don't need to touch lots of existing code as you would just replace the run_command(..) call by an add_task(..). The threading pool would abstract away the whole parallelism and its problems. It would also have the backpressure to the main program, add_task would just not return if there was no place left in the queue. [1] http://www.spinics.net/lists/git/msg258435.html Any feedback welcome! Thanks, Stefan Jonathan Nieder (1): Sending "Fetching submodule <foo>" output to stderr Stefan Beller (1): fetch: fetch submodules in parallel Documentation/fetch-options.txt | 7 ++ builtin/fetch.c | 6 +- builtin/pull.c | 6 ++ run-command.c | 144 ++++++++++++++++++++++++++++++++++++++++ run-command.h | 29 ++++++++ strbuf.c | 31 +++++++++ strbuf.h | 1 + submodule.c | 99 +++++++++++++++++---------- submodule.h | 2 +- t/t0061-run-command.sh | 16 +++++ t/t5526-fetch-submodules.sh | 70 ++++++++++++------- test-run-command.c | 23 +++++++ 12 files changed, 373 insertions(+), 61 deletions(-) -- 2.6.0.rc0.131.gf624c3d -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html