When I was looking at the branches of Jens for work done on submodules not yet upstream I found a commit "WIP threaded submodule fetching[1], and I was side tracked wanting to present a different approach to that. The first patch is a bit unrelated as it relates to the rewrite of git-submodule.sh but also has code in submodule.c and the following patches modify code just around that, so I did not remove that patch from this series. It is the same I sent yesterday. The next patch 2/4 presents a framework for parallel threaded work. It allows to setup a worker pool of <n> threads and then have a queue of tasks which are worked on by the threads. The patch is a the one which I'd request most comments on as I think that can be reused in a variety of situations (parallel checkout of files, parallel fetch of different remotes, or such). I consider the third patch farely boring as it adds argv_array_copy, so I would not expect much discussion there. The last patch 4/4 presents the new workdispatcher from 2/4 in use with just one unsolved problem of how to handle the output of the parallel commands to stdout and stderr. It may be useful to put handling of parallel outputs into the work dispatcher. [1] https://github.com/jlehmann/git-submod-enhancements/commit/47597753206d40e234a47392e258065c9489e2b3 This series applies on top of origin/sb/submodule-helper (d2c6c09ac819, submodule: implement `module_list` as a builtin helper) and can also be found at https://github.com/stefanbeller/git/tree/parallel-submodule-fetch Stefan Beller (4): submodule: implement `module_name` as a builtin helper Add a workdispatcher to get work done in parallel argv_array: add argv_array_clone to clone an existing argv array submodule: add infrastructure to fetch submodules in parallel Makefile | 1 + argv-array.c | 13 ++++ argv-array.h | 1 + builtin/fetch.c | 3 +- builtin/submodule--helper.c | 23 ++++++ git-submodule.sh | 32 ++------ submodule.c | 92 ++++++++++++++++------ submodule.h | 3 +- workdispatcher.c | 184 ++++++++++++++++++++++++++++++++++++++++++++ workdispatcher.h | 29 +++++++ 10 files changed, 332 insertions(+), 49 deletions(-) create mode 100644 workdispatcher.c create mode 100644 workdispatcher.h -- 2.5.0.239.g9728e1d.dirty -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html