On Tue, Nov 9, 2021 at 1:18 PM Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> wrote: > > > On Mon, Nov 08 2021, Taylor Blau wrote: > > > I was discussing this with Elijah today in IRC. I thought that I sent > > the following message to the list, but somehow dropped it from the CC > > list, and only sent it to Elijah and Johannes. > > > > Here it is in its entirety, this time copying the list. > > > > n Thu, Oct 21, 2021 at 01:56:06PM +0200, Johannes Schindelin wrote: > >> 5. The challenge is not necessarily the technical challenges, but the UX for > >> server tools that live “above” the git executable. > >> > >> 1. What kind of output is needed? Machine-readable error messages? > >> > >> 2. What Git objects must be created: a tree? A commit? > >> > >> 3. How to handle, report, and store conflicts? Index is not typically > >> available on the server. > > > > I looked a little bit more into what GitHub would need in order to make > > the switch. For background, we currently perform merges and rebases > > using libgit2 as the backend, for the obvious reason which is that in a > > pre-ORT world we could not write an intermediate result without having > > an index around. > > > > (As a fun aside, we used to expand our bare copy of a repository into a > > temporary working directory, perform the merge there, and then delete > > the directory. We definitely don't do that anymore ;)). > > > > It looks like our current libgit2 usage more-or-less returns an > > (object_id, list<file>) tuple, where: > > > > - a non-NULL object_id is the result of a successful (i.e., > > conflict-free) merge; specifically the oid of the resulting root > > tree > > > > - a NULL object_id and a non-empty list of files indicates that the > > merge could not be completed without manual conflict resolution, and > > the list of files indicates where the conflicts were > > > > When we try to process a conflicted merge, we display the list of files > > where conflicts were present in the web UI. We do have a UI to resolve > > conflicts, but we populate the contents of that UI by telling libgit2 to > > perform the same merge on *just that file*, and writing out the file > > with its conflict markers as the result (and sending that result out to > > a web editor). > > > > So I think an ORT-powered server-side merge would have to be able to: > > > > - write out the contents of a merge (with a tree, not a commit), and > > indicate whether or not that merge was successful with an exit code > > > > - write out the list of files that had conflicts upon failure > > > > Given my limited knowledge of the ORT implementation, it seems like > > writing out the conflicts themselves would be pretty easy. But GitHub > > probably wouldn't use it, or at least not immediately, since we rely > > heavily on being able to recreate the conflicts file-by-file as they are > > needed. > > > > Anyway, I happened to be looking into all of this during the summit, but > > never wrote any of it down. So I figured that this might be helpful in > > case folks are interested in pursuing this further. If so, let me know > > if there are any other questions about what GitHub might want on the > > backend, and I'll try to answer as best I can. > > That's very informative, thanks. Yeah, thanks! > Not that "ort" won't me much better at this, I think the optimizations in "ort" could still be useful. Wouldn't it be nice if rename detection was optimized for example? > but FWIW git-merge-tree > sort of gets most of the way-ish to what you're describing already in > terms of a command interface. Yeah, but if the engine is not up to date, I am not sure it's worth it to reuse it just for the current very limited command interface. > I.e. I'm not the first or last to have (not for anything serious) > implement a dry-run bare-repo merge with something like: > > git merge-tree origin/master git-for-windows/main origin/seen >diff > # Better regex needed, but basically this > grep "^\+<<<<<<< \.our$" diff && conflict=t > > So with some parsing of that command output you can get a diff with one > side or the other applied. Yeah, it looks like it would be easy to add options like --ours, --theirs, etc, to get only the part we are interested in. And we already easily see if the merge conflicted or not from the current output, as it seems to output: "0 mode sha1 filename" in case of a successful merge, and: "1 mode sha1 filename" "2 mode sha1 filename" "3 mode sha1 filename" in case of conflicts. > From there it's a matter of applying the patch, and from there you'd get > blobs/trees. which is painful from just having a diff & no index, so > it's a common use-case of libgit2 for just such basic usage. > > But to the extent that we were talking about plumbing interfaces > wouldn't basically a git-merge-tree on steroids (or extension thereof) > do, i.e.: > > * Ask it to merge X heads, returns whether it worked or not > * ... and can return a diff with conflict markers like this > * ... for just some <pathspec> > * ... maybe with the conflict already "resolved" one way or the other? > * ... optionally, after some markers write one/both sides, spew out the > relevant tree/blob OIDs > * ... which again, could be limited by the <pathspec> above. > > I'm thinking of something that basically works like git for-each-ref --format="". So: > > git merge-tree --format="..." <heads> -- <pathspec> > > Where that <format> can be custom \0-delimited (or whatever) sections of > payload that could have whatever combination of the above you'd need. I > think git-for-each-ref is probably the best example we've got of a > plumbing interface in this category, i.e. being able to extract > arbitrary payloads via format specifiers & "path" (well, ref) > limitation. The current synopsis is: git merge-tree <base-tree> <branch1> <branch2> which is quite different from what you are proposing. Given that it seems worth it to use a different underlying engine (actually the "ort" one) than the current one, I think that it might be better to start from scratch with a new command using the "ort" engine. > Elijah probably has much better ideas already, I'm just spitballing. Yeah, I'd be interested in knowing Elijah's opinion on this. Although maybe I misunderstood, but I thought that Elijah had plans to send patches related to this to the list after v2.34. > But if something like that worked it would be mostly a matter of > stealing code from for-each-ref and the like, and then the <handwaiving> > mapping that to ORT callbacks somehow. Yeah, but what would be left from the original git merge-tree then? Wouldn't it make more sense to start with a new command that has roughly the same features as git merge-tree and a similar interface (though maybe not quite the same as we could anticipate some future extensions and maybe learn a bit from other commands), but uses "ort". Then we could grow it as we want, without being burdened by the git merge-tree legacy, in the same way as "ort" was developed without being burdened by the recursive merge legacy? > And then it could even learn a --batch mode, which with those formats > could allow calling it without paying the price for command > re-invocation, something like the update-ref/proposed cat-file interface > discussed in another thread at [1]. Yeah, sure.
