So here is the next iteration of the recursive fetching for submodules.
Changes to the first patchset and its fixes:
* Squashed all fixes into the first patch
* Added the new config setting 'fetch.recursive' to set the default
for all submodules
IMO this patchset should replace the one currently in pu.
Still unresolved issues:
1) Performance
Kevin reported the fetch time went up from 1.5s to 20s for him
because of the recursion. Kevin, could you please test the branch
"parallel-submodule-fetch" from my github repository:
http://github.com/jlehmann/git-submod-enhancements.git
It has these three patches based on next plus a preliminary
commit fetching submodules in parallel (but note that a limit
of 128 submodules is hardcoded and the output might be mixed
between the fetch threads, I'll fix that when you confirm the
performance benefit I expect).
(It took me some time to get that working, the DNS server in my
DSL router seems to silently drop IPv6 requests. This lead to a
15 second timeout in getaddrinfo() when forking fetch commands
in quick succession, turning the intended speedup into a
massive slowdown. If you happen to run into the same issue,
please try the "broken-dns-server-hack" branch which contains a
workaround for that issue).
2) The proper default
Kevin argues that submodule fetching should only happen when new
commits for the submodule have been fetched (which is kind of
similar to what "git submodule update" does now), while I argue
that submodules should always be fetched by default, no matter
if new commits from submodules are committed inside the
superproject or not.
Ok, lets look at the basic ways you can run the fetch command and
look at possible defaults:
a) "git fetch --all"
The user wanted to fetch new commits from all remotes. I think
git should also fetch all submodules then, no matter if new
commits from them are fetched in the superproject, as the user
explicitly said he wants everything. Objections?
b) "git fetch [<repository>]"
The user wants to fetch from the default [or a single repo]. I
think all submodules should be fetched too, Kevin thinks this
should happen only when it is necessary (at least for his 'H'
repository). While I think fetching all submodules too is
consistent with the fact that you get all branches in the
superproject too, whether you need them or not, we can't seem
to agree on this one (also see my proposal below).
c) "git fetch [<repository>] <refspec>"
If the user wants to fetch only commits from a certain refspec,
I think that it is sane to let git fetch only those submodules
where new commits were fetched in the superproject as he was
explicitly asking only for a subset of available commits.
Assuming we agree on a) and - the still to be implemented - c), It
looks like we need a new config setting for 'fetch.recursive' and
'submodule.<name>.fetch', say 'changed'? This would allow Kevin to
set 'submodule.H.fetch' to 'changed' in the .gitmodules describing
it. Then "git fetch" would only then recurse further when new
commits from 'H' have been used in the commits fetched in its
superproject. Would that be an acceptable solution?
Jens Lehmann (3):
fetch/pull: Recursively fetch populated submodules
Submodules: Add the new "fetch" config option for fetch and pull
Add the 'fetch.recursive' config setting
Documentation/config.txt | 10 ++
Documentation/fetch-options.txt | 15 +++
Documentation/gitmodules.txt | 8 ++
builtin/fetch.c | 58 ++++++++---
git-pull.sh | 10 ++-
submodule.c | 91 +++++++++++++++++-
submodule.h | 3 +
t/t5526-fetch-submodules.sh | 209 +++++++++++++++++++++++++++++++++++++++
t/t7403-submodule-sync.sh | 2 +-
9 files changed, 388 insertions(+), 18 deletions(-)
create mode 100755 t/t5526-fetch-submodules.sh
--
1.7.3.1.108.gb6303
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html