On Mon, Nov 14 2022, Glen Choo wrote: > Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> writes: > >> The "--super-prefix" facility was introduced in [1] has always been a >> transitory hack, which is why we've made it an error to supply it as >> an option to "git" to commands that don't know about it. >> >> That's been a good goal, as it has a global effect we haven't wanted >> calls to get_super_prefix() from built-ins we didn't expect. >> >> But it has meant that when we've had chains of different built-ins >> using it all of the processes in that "chain" have needed to support >> it, and worse processes that don't need it have needed to ask for >> "SUPPORT_SUPER_PREFIX" because their parent process needs it. >> >> That's how "fsmonitor--daemon" ended up with it, per [2] it's called >> from (among other things) "submodule--helper absorbgitdirs", but as we >> declared "submodule--helper" as "SUPPORT_SUPER_PREFIX" we needed to >> declare "fsmonitor--daemon" as accepting it too, even though it >> doesn't care about it. >> >> But in the case of "absorbgitdirs" it only needed "--super-prefix" to >> invoke itself recursively, and we'd never have another "in-between" >> process in the chain. So we didn't need the bigger hammer of "git >> --super-prefix", and the "setenv(GIT_SUPER_PREFIX_ENVIRONMENT, ...)" >> that it entails. >> >> Let's instead accept a hidden "--super-prefix" option to >> "submodule--helper absorbgitdirs" itself. >> >> Eventually (as with all other "--super-prefix" users) we'll want to >> clean this code up so that this all happens in-process. I.e. needing >> any variant of "--super-prefix" is itself a hack around our various >> global state, and implicit reliance on "the_repository". This stepping >> stone makes such an eventual change easier, as we'll need to deal with >> less global state at that point. >> >> The "fsmonitor--daemon" test adjusted here was added in [3]. The >> comment added in that commit has been out-of-date from the beginning, >> and the "have_t2_error_event()" was being overly specific in testing >> for a bug that we *don't* have. Let's instead test for the stdout and >> stderr that we *do have*. > > I didn't understand this bit initially, because I read this as > "have_t2_error_event() isn't catching bugs", which isn't true. But I see > what you mean after inspecting the test_cmp output: > > @@ -1 +1,2 @@ > Migrating git directory of 'dir_1/dir_2/sub' from 'dir_1/dir_2/sub/.git' to '.git/modules/dir_1/dir_2/sub' > +fatal: fsmonitor--daemon doesn't support --super-prefix > > IOW, it doesn't make sense to inspecting the tr2 output for stray > warnings when our stderr is so obviously broken. But at the end of the > series, I don't think we even need this test at all because if we don't > have a global "--super-prefix", there's literally no reason for > fsmonitor--daemon to worry about absorbgitdirs or "supporting" the super > prefix. That's why I removed it in [1]. > > Wondering aloud, that stderr suggests that the "git submodule absorbgitdirs" > invocation failed. I wonder why the test didn't catch a bad exit code > (and resulting in us inspecting stderr unnecessarily). I think it's not > worth looking too closely at right now, but we could revisit this after > we get builtin/submodule.c. > > [1] https://lore.kernel.org/git/20221109004708.97668-5-chooglen@xxxxxxxxxx I'm prepping a re-roll where I'll keep some of this test, as testing fsmonitor+absorbgitdirs is worthhwile, but the commit message etc. is rephrased. >> diff --git a/git.c b/git.c >> index 10202a7f126..b1b7e1a837e 100644 >> --- a/git.c >> +++ b/git.c >> @@ -539,7 +539,7 @@ static struct cmd_struct commands[] = { >> { "format-patch", cmd_format_patch, RUN_SETUP }, >> { "fsck", cmd_fsck, RUN_SETUP }, >> { "fsck-objects", cmd_fsck, RUN_SETUP }, >> - { "fsmonitor--daemon", cmd_fsmonitor__daemon, SUPPORT_SUPER_PREFIX | RUN_SETUP }, >> + { "fsmonitor--daemon", cmd_fsmonitor__daemon, RUN_SETUP }, >> { "gc", cmd_gc, RUN_SETUP }, >> { "get-tar-commit-id", cmd_get_tar_commit_id, NO_PARSEOPT }, >> { "grep", cmd_grep, RUN_SETUP_GENTLY }, >> diff --git a/parse-options.h b/parse-options.h >> index b6ef86e0d15..50d852f2991 100644 >> --- a/parse-options.h >> +++ b/parse-options.h >> @@ -369,6 +369,10 @@ int parse_opt_tracking_mode(const struct option *, const char *, int); >> { OPTION_CALLBACK, 0, "abbrev", (var), N_("n"), \ >> N_("use <n> digits to display object names"), \ >> PARSE_OPT_OPTARG, &parse_opt_abbrev_cb, 0 } >> +#define OPT__SUPER_PREFIX(var) \ >> + OPT_STRING_F(0, "super-prefix", (var), N_("prefix"), \ >> + N_("prefixed path to initial superproject"), PARSE_OPT_HIDDEN) >> + > > Could we default to "" instead of NULL? (possibly via a callback). I > think there's never any good reason to have NULL instead of "", e.g. > since this is internal, we don't care to distinguish between > "--super-prefix=''" and not passing "--super-prefix" at all... I saw this comment on another patch, I think I'll keep it for now, but I updated the reason: It's because the "prefix" is either NULL or a non-zero length string, so we're doing the same here to be consistent with it. I think that makes sense, but ultimately it's an arbitrary choice, but as they're sibling variables in the cases they're used doing the same seems worthwhile... >> #define OPT__COLOR(var, h) \ >> OPT_COLOR_FLAG(0, "color", (var), (h)) >> #define OPT_COLUMN(s, l, v, h) \ >> diff --git a/submodule.c b/submodule.c >> index c47358097fd..d9fd0af81b6 100644 >> --- a/submodule.c >> +++ b/submodule.c >> @@ -2268,7 +2268,8 @@ int validate_submodule_git_dir(char *git_dir, const char *submodule_name) >> * Embeds a single submodules git directory into the superprojects git dir, >> * non recursively. >> */ >> -static void relocate_single_git_dir_into_superproject(const char *path) >> +static void relocate_single_git_dir_into_superproject(const char *path, >> + const char *super_prefix) >> { >> char *old_git_dir = NULL, *real_old_git_dir = NULL, *real_new_git_dir = NULL; >> struct strbuf new_gitdir = STRBUF_INIT; >> @@ -2302,7 +2303,7 @@ static void relocate_single_git_dir_into_superproject(const char *path) >> real_old_git_dir[off] == real_new_git_dir[off]) >> off++; >> fprintf(stderr, _("Migrating git directory of '%s%s' from '%s' to '%s'\n"), >> - get_super_prefix_or_empty(), path, >> + (super_prefix ? super_prefix : ""), path, >> real_old_git_dir + off, real_new_git_dir + off); > > > Which would make sites like these a bit cleaner. > > Tangentially, 'default to "" if NULL' sounds like a common pattern. Is > there a good reason not to have a macro or inline function to do that? > e.g. we think the ternary expression is good enough?) See above. >> >> relocate_gitdir(path, real_old_git_dir, real_new_git_dir); >> @@ -2313,7 +2314,8 @@ static void relocate_single_git_dir_into_superproject(const char *path) >> strbuf_release(&new_gitdir); >> } >> >> -static void absorb_git_dir_into_superproject_recurse(const char *path) >> +static void absorb_git_dir_into_superproject_recurse(const char *path, >> + const char *super_prefix) >> { >> >> struct child_process cp = CHILD_PROCESS_INIT; >> @@ -2321,10 +2323,11 @@ static void absorb_git_dir_into_superproject_recurse(const char *path) >> cp.dir = path; >> cp.git_cmd = 1; >> cp.no_stdin = 1; >> - strvec_pushf(&cp.args, "--super-prefix=%s%s/", >> - get_super_prefix_or_empty(), path); >> strvec_pushl(&cp.args, "submodule--helper", >> "absorbgitdirs", NULL); >> + strvec_pushf(&cp.args, "--super-prefix=%s%s/", super_prefix ? >> + super_prefix : "", path); >> + >> prepare_submodule_repo_env(&cp.env); >> if (run_command(&cp)) >> die(_("could not recurse into submodule '%s'"), path); >> @@ -2335,7 +2338,8 @@ static void absorb_git_dir_into_superproject_recurse(const char *path) >> * having its git directory within the working tree to the git dir nested >> * in its superprojects git dir under modules/. >> */ >> -void absorb_git_dir_into_superproject(const char *path) >> +void absorb_git_dir_into_superproject_sp(const char *path, >> + const char *super_prefix) >> { >> int err_code; >> const char *sub_git_dir; >> @@ -2377,14 +2381,14 @@ void absorb_git_dir_into_superproject(const char *path) >> char *real_common_git_dir = real_pathdup(get_git_common_dir(), 1); >> >> if (!starts_with(real_sub_git_dir, real_common_git_dir)) >> - relocate_single_git_dir_into_superproject(path); >> + relocate_single_git_dir_into_superproject(path, super_prefix); >> >> free(real_sub_git_dir); >> free(real_common_git_dir); >> } >> strbuf_release(&gitdir); >> >> - absorb_git_dir_into_superproject_recurse(path); >> + absorb_git_dir_into_superproject_recurse(path, super_prefix); >> } >> >> int get_superproject_working_tree(struct strbuf *buf) >> diff --git a/submodule.h b/submodule.h >> index b52a4ff1e73..e5ee13fb06a 100644 >> --- a/submodule.h >> +++ b/submodule.h >> @@ -164,7 +164,12 @@ void submodule_unset_core_worktree(const struct submodule *sub); >> */ >> void prepare_submodule_repo_env(struct strvec *env); >> >> -void absorb_git_dir_into_superproject(const char *path); >> +void absorb_git_dir_into_superproject_sp(const char *path, >> + const char *super_prefix); >> +static inline void absorb_git_dir_into_superproject(const char *path) >> +{ >> + absorb_git_dir_into_superproject_sp(path, NULL); >> +} > > Is there a reason you chose to go with _sp() instead of changing the > original function signature? I tested out that change, and it seems > rather small (absorb_git_dir_into_superproject() only has 4 call sites). I've just changed that, so no more *_sp()... > Crucially, changing the signature catches a "git read-tree" call site: > > diff --git a/submodule.c b/submodule.c > index d9fd0af81b..e79a04d3e3 100644 > --- a/submodule.c > +++ b/submodule.c > @@ -2139,7 +2139,9 @@ int submodule_move_head(const char *path, > if (!(flags & SUBMODULE_MOVE_HEAD_DRY_RUN)) { > if (old_head) { > if (!submodule_uses_gitfile(path)) > - absorb_git_dir_into_superproject(path); > + /* Pass super_prefix properly later. */ > + absorb_git_dir_into_superproject(path, > + get_super_prefix()); > } else { > struct strbuf gitdir = STRBUF_INIT; > submodule_name_to_gitdir(&gitdir, the_repository, > > which would otherwise be broken since we used to read the global super > prefix, but we don't do that in this patch. This has no test coverage > (bleh), but it shouldn't be too hard, something like: > > git init super && > cd super && > # Create an unabsorbed submodule right in the worktree > git init sub && > test_commit -C sub "foo" && > git add sub && > git commit -m "Add submodule" && > test_commit "bar" && > # This should invoke "git read-tree" and absorb the git dir. > # Or maybe we should invoke "git read-tree" directly? > git checkout --recurse-submodules HEAD^ 2>err && > # Search for the shortened message > grep "Migrating submodule sub from sub to .git/modules/...." err Ah! Yeah that is a good catch, and a great reason to do it this way. I tried to come up with a test case that actually tested this. I.e. I set a "here = 1" at that absorb_git_dir_into_superproject() in the diff above, and then if BUG()'d out in that function, but the part where we're about to recurse. I couldn't get it to trigger, but I didn't look at it very long. I.e. doesn't this require not this sort of test, but something where it's being absorbed, *and* we'd walk and absorb it recursiveyl? At least with checkout we then run into an error (can't remember the message, sorry, this was earlier), and I haven't found a way to trigger it. But I'll poke it some more, but if you happen to have a test case for it that would really help...
