This build on top of origin/sb/submodule-helper Patches 1-8 are parallelising fetching submodules. Thanks Eric, Jeff and Junio for the discussion and review! Based on your feedback: Patch 2 is optional and may be spun out as its own mini series. Instead of spinning we can poll until data are ready on the fd. Patch 3,4 have been revamped to include your suggestions, having xread_nonblock and strbuf_read_once as their own functions not to be integrated into other existing functions. Patch 5 was fixed to return 0 or -1 in the error case. Patch 6 has major syntactically changes such as no slots any more, but in_use flags. The return code was flipped (1 means there is a child to be executed, while 0 means we're done). Patch 7,8 convert git fetch --recuse-submodules to use the fruit of patch 6. Patches 9 an onwards are RFC, so I could spin them out as their own series as they convert parts of "submodule update" to C using the new asynchronous run command code. Patch 9 adds the update strategy to the submodule cache Patches 10,11,12 are just moving code around to make cmd_update leaner. Patch 13 shows how I plan to parallelize updating the submodules for now. Any feedback welcome, Thanks, Stefan Jonathan Nieder (1): Sending "Fetching submodule <foo>" output to stderr Stefan Beller (13): xread: poll on non blocking fds xread_nonblock: add functionality to read from fds nonblockingly strbuf: add strbuf_read_once to read without blocking run-command: factor out return value computation run-command: add an asynchronous parallel child processor fetch_populated_submodules: use new parallel job processing submodules: allow parallel fetching, add tests and documentation submodule config: keep update strategy around git submodule update: pass --prefix only with a non empty prefix git submodule update: cmd_update_recursive git submodule update: cmd_update_clone git submodule update: cmd_update_fetch Rewrite submodule update in C Documentation/fetch-options.txt | 7 + builtin/fetch.c | 6 +- builtin/pull.c | 6 + builtin/submodule--helper.c | 247 +++++++++++++++++++++++++++++ git-compat-util.h | 1 + git-submodule.sh | 339 ++++++++++++++-------------------------- run-command.c | 329 +++++++++++++++++++++++++++++++++++--- run-command.h | 38 +++++ strbuf.c | 11 ++ strbuf.h | 6 + submodule-config.c | 16 ++ submodule-config.h | 1 + submodule.c | 119 +++++++++----- submodule.h | 2 +- t/t0061-run-command.sh | 20 +++ t/t5526-fetch-submodules.sh | 70 ++++++--- test-run-command.c | 24 +++ wrapper.c | 41 ++++- 18 files changed, 972 insertions(+), 311 deletions(-) -- 2.5.0.275.ge015d2a -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html