On Wed, Apr 14, 2021 at 12:32:17PM +0200, Ævar Arnfjörð Bjarmason wrote: > On Fri, Apr 09 2021, Emily Shaffer wrote: > > > I'm hoping to work on some other submodule-centric stuff over the coming > > months, and it might end up being very useful to be able to tell "am I a > > submodule?" and "how do I talk to my superproject?" more generally - so > > I'm really open to figuring out a better way than this, if folks have > > ideas. > > > > Patch 1 is a small refactor that we can take or leave - I found > > "SCOPE_SUBMODULE" to be pretty ambiguous, especially since it seems to > > refer to configs from .gitmodules. Even though I decided that > > "superproject" was a better name than "submodule" I still wasn't super > > happy with the ambiguity. But we can drop it if folks don't want to > > rename. > > This is less on your patch, and more on the larger work you're > suggesting, but the two are kind of related. Skip to the paragraph > starting with "But why" below for the relevance :) > > I very much wish that we could eventually make the use of submodules > totally transparent, i.e. (taking the example of git.git): > > * You clone, and we just get objects from > https://github.com/cr-marcstevens/sha1collisiondetection.git too > > * The fact that we have: > > 160000 commit 855827c583bc30645ba427885caa40c5b81764d2 sha1collisiondetection > > Would become totally invisible to most users unless they run some > gutsy ls-tree/files comand. > > We used to have a full git dir at sha1collisiondetection/.git and all > the UX issues that entailed (e.g. switching to an old commit without > the submodule). > > Now it's a stub and the actual repo is at > .git/modules/sha1collisiondetection/, so we're kind of partially > there. Side note: when I was writing the tests for patch 2 in this series I noticed it was still really easy to end up with a full git dir at e.g. sha1collisiondetection/.git, if you are trying to create a new repo to use as a submodule (easily could be the case when working on a "greenfield" project and you're the original author). There is definitely a reason that I copied the (IMO) hack from the other submodule test suite using the trash directory as a remote for my new submodule. ;) I wonder whether I was just doing it wrong, or if we need some established flow (maybe with `git submodule` subcommand) to create a brand new submodule, not cloned from somewhere, and put its gitdir inside of .git/modules? > * I would think that the next (but big) logical step would be to use > some combination of delta islands, upcoming sparse indexes etc. to > actually share the object stores of the parent and submodule. Interesting - I'm trying to think of reasons not to and coming up blank, but I also don't have much firsthand experience with the area of the code that looks through the object store, so what do I know? > Things like "git fsck" which now just punt on COMMIT would need to > become smarter, but e.g. we could repack (or not, with islands) > between parent and submodule. > > I would think that this end goal makes more sense than the current > status quo of teaching every command that needs to e.g. grep the tree to > have a "--recurse-submodules". The distinction would be invisible to the > likes of "git-grep". Yeah, I see where you're going, I think. Teaching everyone --recurse-submodules or to respect the config setting (core.recurseSubmodules? whatever it is) is inherently fragile, since it relies on human reviewers to remember to chide patch authors to think of the submodules use case. Neat. > It would mean more complexity in e.g. "git commit", but we can imagine > if you wanted a cross-submodule commit it could do those commits > recursively, update parent COMMIT entries etc. (and even, optionally, > push out the submodule changes). That particular thing being so ad-hoc > is a *very* frequent pain point in submodule use. Yeah, it sounds like you're describing the approach I was hoping to use for commit-with-recursion: - Note each submodule with staged changes, as well as the superproject - (Optional? but might be nice) Open an editor with all the commit messages separated by scissors, so you can easily refer back to or modify the submodule commit messages while writing the superproject commit message - Generate all the submodule commits with the supplied commit-msgs - Take the commit IDs of all the newly created commits, stage them in the superproject, and generate the superproject commit with the supplied commit-msg Maybe the editor bit is too much, but Jonathan Nieder at least really liked that idea :) But the bit you're talking about - generating the submodules first and then staging and committing in the superproject "on the fly" - was the approach I was hoping to take. > > But why am I talking about this here when all you're suggesting is > another config level? > > Well, I think (but have not carefully thought about) that this > CONFIG_SCOPE_GITMODULES is probably a narrow net improvement now. If you > set most options in your .git/config to you that's the same logical > project, why shouldn't you get your diff setting or whatever because you > cd'd to a submodule "in the same project" (from the view of the user). > > But I think that for a wider "improve submodules" effort it's worth > someone (and right now, that sounds like it's you) thinking about where > we're going with the feature. Maybe with some technical doc identifying > the most common pain points, what we propose (or could envision) doing > about them. > > So e.g. in this case, having per-submodule config could be a step > forward, but it could also be one more step of walking in a circle. > > I.e. don't think any user asked for or wanted to stitch together > multiple .git directories into one linked pseudo-checkout, that's > ultimately something we're exposing as an implementation detail. If we > no longer expose that implementation detail, would we be stuck > supporting what's ultimately a workaround feature? > > None of that means we shouldn't have that one step forward that solves > real problems today. > > But I think we should think about the end goal(s) sooner than > later. Yeah, this is actually a good nudge for me. Internally we've got a big nice doc explaining all our submodule plans for the next 6-9 months - but I should probably get to sharing that with the list ;) I'd say to look for it either this Friday or next Friday. > E.g. in your case, do you *really* want another config level, or > is it just the easiest way to get what you actually want, which is for a > "git config" in the submodule dir to perhaps consider its .git/config > and .git/modules/sha1collisiondetection/config as the same file for the > purposes of config parsing? Sans things like the remote URLs etc. As for this specific case, I want what is in the patch. Using a new config file doesn't feel like a compromise to me - I actually would prefer users to be able to explicitly choose shared vs. repo-specific configs, rather than for we Git devs to implicitly decide which configs are fine to share and which aren't. (I could also see having explicitly shared or non-shared configs making it easier for wrappers to leverage the Git config infrastructure, without mirroring our own "list of configs to not share to submodule" for themselves.) This RFC is mostly here to enable shared hooks, as you might have guessed - but even with hooks, it's easy to imagine wanting a blend of inherited vs. per-repo hooks. For example, I want to inherit a hook to create a Gerrit Change-Id footer in my superproject and all my submodules, definitely - but if my superproject is written in C and includes a submodule which is in, I dunno, Rust or Zig or Perl or whatever people are writing these days, I definitely don't want to try and run my C linter from my superproject on my 15 Rust submodules - and I definitely don't want to disable it in each one. - Emily
