On Thu, Aug 6, 2015 at 1:08 PM, Jens Lehmann <Jens.Lehmann@xxxxxx> wrote: > Am 06.08.2015 um 19:35 schrieb Stefan Beller: >> >> When I was looking at the branches of Jens for work done on submodules >> not yet upstream I found a commit "WIP threaded submodule fetching[1], >> and I was side tracked wanting to present a different approach to that. > > > Cool. I didn't follow that route further than building a proof of > concept because I ran into a nasty DNS-timeout on my router at home > and at work we host all repos on a not-so-beefy server making parallel > fetch rather pointless. But I suspect this approach will bring down > fetch times for some users. The difference between your proof of concept and mine is to have the number of threads easier configurable. (Think of "git fetch --recurse-submodules=yes -j4", similar to "make -j4" splitting work up to 4 different programs at the same time. And that thing would be possible to add in this series) If you fetch lots of submodules, both the client and server load should come in an ping-pong on-off pattern as the client waits for the server to prepare stuff and get it sent to it and then the client needs time to resolve deltas and write to disk. Depending on the duty cycle of each, a different number of parallel threads make sense (I would expected that they shift their phases against each other by pure randomness, i.e. one thread is currently resolving deltas while the other thread is telling the server to get some work done, so both the client and server get utilized at the same time). > > Maybe we could also re-use parallel fetch for multiple upstreams in > the superproject when doing a "git fetch --all" without too much > extra work? That's why I tried advertising RFC patch 2/4 as I believe it could be easily reused for such things like "git fetch --all", but maybe other people see problems with it (over/under engineered, wrong things added, but other critical things missing) so I'd like to hear opinions on that. :) -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html