Here are some patches to add a --remerge-diff capability to show & log,
which works by comparing merge commits to an automatic remerge (note that
the automatic remerge tree can contain files with conflict markers).
Changes since v4:
* Just a few minor tweaks -- comment on output being subject to change,
simpler test skipping check, and improved bug message
Changes since v3:
* Filter conflict headers according to pathspecs
* Instead of always including conflict headers for all diff types, only
select them with --diff-filter=U OR whenever the associated diff in
question is selected
* New testcases dealing with --diff-filter, pathspecs, and default history
simplification
* Switched back from die_errno() to die()
Changes NOT included (mostly because I'm not sure what to add or where):
* Johannes Altimanninger suggested changing the ordering of the new headers
relative to other headers. He made a good point, but I also like having
the conflict messages next to the text, so I'm conflicted about what's
best.
* (Technically not part of this feature, but kind of related.) Months ago,
Junio suggested documenting ${GIT_DIR}/AUTO_MERGE better
(https://lore.kernel.org/git/xmqqtuj4nepe.fsf@gitster.g/). I looked at
the time, but couldn't find a place to put it that made sense to me.
Changes since v2 (of the restarted submission):
* Numerous small improvements suggested by Johannes Altmanninger
* Avoid including conflict messages from inner merges (due to example
pointed out by Ævar).
* Added a "remerge" prefix to all the new diff headers (suggested by Junio
in a previous round, but I couldn't come up with a good name before. It
suddenly hit me that "remerge" is an obvious prefix to use, and even
helps explain what the rest of the line is for.)
Changes since v1 (of the restarted submission, which technically was v2):
* Restructured the series, so the first patch introduces the feature --
with a bunch of caveats. Subsequent patches clean up those caveats. This
avoids introducing not-yet-used functions, and hopefully makes review
easier.
* added testcases
* numerous small improvements suggested by Ævar and Junio
Changes since original submission[1]:
* Rebased on top of the version of ns/tmp-objdir that Neeraj submitted
(Neeraj's patches were based on v2.34, but ns/tmp-objdir got applied on
an old commit and does not even build because of that).
* Modify ll-merge API to return a status, instead of printing "Cannot merge
binary files" on stdout[2] (as suggested by Peff)
* Make conflict messages and other such warnings into diff headers of the
subsequent remerge-diff rather than appearing in the diff as file content
of some funny looking filenames (as suggested by Peff[3] and Junio[4])
* Sergey ack'ed the diff-merges.c portion of the patches, but that wasn't
limited to one patch so not sure where to record that ack.
[1]
https://lore.kernel.org/git/pull.1080.git.git.1630376800.gitgitgadget@xxxxxxxxx/;
GitHub wouldn't let me change the target branch for the PR, so I had to
create a new one with the new base and thus the reason for not sending this
as v2 even though it is. [2]
https://lore.kernel.org/git/YVOZRhWttzF18Xql@xxxxxxxxxxxxxxxxxxxxxxx/,
https://lore.kernel.org/git/YVOZty9D7NRbzhE5@xxxxxxxxxxxxxxxxxxxxxxx/ [3]
https://lore.kernel.org/git/YVOXPTjsp9lrxmS6@xxxxxxxxxxxxxxxxxxxxxxx/ [4]
https://lore.kernel.org/git/xmqqr1d7e4ug.fsf@gitster.g/
=== FURTHER BACKGROUND (original cover letter material) ==
Here are some example commits you can try this out on (with git show
--remerge-diff $COMMIT):
* git.git conflicted merge: 07601b5b36
* git.git non-conflicted change: bf04590ecd
* linux.git conflicted merge: eab3540562fb
* linux.git non-conflicted change: 223cea6a4f05
Many more can be found by just running git log --merges --remerge-diff in
your repository of choice and searching for diffs (most merges tend to be
clean and unmodified and thus produce no diff but a search of '^diff' in the
log output tends to find the examples nicely).
Some basic high level details about this new option:
* This option is most naturally compared to --cc, though the output seems
to be much more understandable to most users than --cc output.
* Since merges are often clean and unmodified, this new option results in
an empty diff for most merges.
* This new option shows things like the removal of conflict markers, which
hunks users picked from the various conflicted sides to keep or remove,
and shows changes made outside of conflict markers (which might reflect
changes needed to resolve semantic conflicts or cleanups of e.g.
compilation warnings or other additional changes an integrator felt
belonged in the merged result).
* This new option does not (currently) work for octopus merges, since
merge-ort is specific to two-parent merges[1].
* This option will not work on a read-only or full filesystem[2].
* We discussed this capability at Git Merge 2020, and one of the
suggestions was doing a periodic git gc --auto during the operation (due
to potential new blobs and trees created during the operation). I found a
way to avoid that; see [2].
* This option is faster than you'd probably expect; it handles 33.5 merge
commits per second in linux.git on my computer; see below.
In regards to the performance point above, the timing for running the
following command:
time git log --min-parents=2 --max-parents=2 $DIFF_FLAG | wc -l
in linux.git (with v5.4 checked out, since my copy of linux is very out of
date) is as follows:
DIFF_FLAG=--cc: 71m 31.536s
DIFF_FLAG=--remerge-diff: 31m 3.170s
Note that there are 62476 merges in this history. Also, output size is:
DIFF_FLAG=--cc: 2169111 lines
DIFF_FLAG=--remerge-diff: 2458020 lines
So roughly the same amount of output as --cc, as you'd expect.
As a side note: git log --remerge-diff, when run in various repositories and
allowed to run all the way back to the beginning(s) of history, is a nice
stress test of sorts for merge-ort. Especially when users run it for you on
their repositories they are working on, whether intentionally or via a bug
in a tool triggering that command to be run unexpectedly. Long story short,
such a bug in an internal tool existed in December 2020 and this command was
run on an internal repository and found a platform-specific bug in merge-ort
on some really old merge commit from that repo. I fixed that bug (a
STABLE_QSORT thing) while upstreaming all the merge-ort patches in the mean
time, but it was nice getting extra testing. Having more folks run this on
their repositories might be useful extra testing of the new merge strategy.
Also, I previously mentioned --remerge-diff-only (a flag to show how
cherry-picks or reverts differ from an automatic cherry-pick or revert, in
addition to showing how merges differ from an automatic merge). This series
does not include the patches to introduce that option; I'll submit them
later.
Two other things that might be interesting but are not included and which I
haven't investigated:
* some mechanism for passing extra merge options through (e.g.
-Xignore-space-change)
* a capability to compare the automatic merge to a second automatic merge
done with different merge options. (Not sure if this would be of interest
to end users, but might be interesting while developing new a
--strategy-option, or maybe checking how changing some default in the
merge algorithm would affect historical merges in various repositories).
[1] I have nebulous ideas of how an Octopus-centric ORT strategy could be
written -- basically, just repeatedly invoking ort and trying to make sure
nested conflicts can be differentiated. For now, though, a simple warning is
printed that octopus merges are not handled and no diff will be shown. [2]
New blobs/trees can be written by the three-way merging step. These are
written to a temporary area (via tmp-objdir.c) under the git object store
that is cleaned up at the end of the operation, with the new loose objects
from the remerge being cleaned up after each individual merge.
Elijah Newren (10):
show, log: provide a --remerge-diff capability
log: clean unneeded objects during `log --remerge-diff`
ll-merge: make callers responsible for showing warnings
merge-ort: capture and print ll-merge warnings in our preferred
fashion
merge-ort: mark a few more conflict messages as omittable
merge-ort: format messages slightly different for use in headers
diff: add ability to insert additional headers for paths
show, log: include conflict/warning messages in --remerge-diff headers
merge-ort: mark conflict/warning messages from inner merges as
omittable
diff-merges: avoid history simplifications when diffing merges
Documentation/diff-options.txt | 14 +-
apply.c | 5 +-
builtin/checkout.c | 12 +-
builtin/log.c | 15 ++
diff-merges.c | 14 ++
diff.c | 124 +++++++++++++-
diff.h | 3 +-
ll-merge.c | 40 +++--
ll-merge.h | 9 +-
log-tree.c | 118 ++++++++++++-
merge-blobs.c | 5 +-
merge-ort.c | 55 ++++++-
merge-ort.h | 10 ++
merge-recursive.c | 9 +-
merge-recursive.h | 2 +
notes-merge.c | 5 +-
rerere.c | 9 +-
revision.h | 6 +-
t/t4069-remerge-diff.sh | 291 +++++++++++++++++++++++++++++++++
t/t6404-recursive-merge.sh | 9 +-
t/t6406-merge-attr.sh | 9 +-
tmp-objdir.c | 5 +
tmp-objdir.h | 6 +
23 files changed, 727 insertions(+), 48 deletions(-)
create mode 100755 t/t4069-remerge-diff.sh
base-commit: 4e44121c2d7bced65e25eb7ec5156290132bec94
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1103%2Fnewren%2Fremerge-diff-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1103/newren/remerge-diff-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1103
Range-diff vs v4:
1: 0b94724311d ! 1: 0a260125266 show, log: provide a --remerge-diff capability
@@ Documentation/diff-options.txt: ifdef::git-log[]
+ create a temporary tree object -- potentially containing files
+ with conflict markers and such. A diff is then shown between
+ that temporary tree and the actual merge commit.
+++
++The output emitted when this option is used is subject to change, and
++so is its interaction with other options (unless explicitly
++documented).
++
--diff-merges=combined:::
--diff-merges=c:::
@@ t/t4069-remerge-diff.sh (new)
+. ./test-lib.sh
+
+# This test is ort-specific
-+test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {
++if test "${GIT_TEST_MERGE_ALGORITHM}" != ort
++then
+ skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+ test_done
-+}
++fi
+
+test_expect_success 'setup basic merges' '
+ test_write_lines 1 2 3 4 5 6 7 8 9 >numbers &&
2: f06de6c1b2f ! 2: ed0d60de24c log: clean unneeded objects during `log --remerge-diff`
@@ log-tree.c: static int do_remerge_diff(struct rev_info *opt,
+ if (opt->remerge_objdir)
+ tmp_objdir_discard_objects(opt->remerge_objdir);
+ else
-+ BUG("unable to remove temporary object directory");
++ BUG("did a remerge diff without remerge_objdir?!?");
return !opt->loginfo;
}
3: 8d6c3d48f0e = 3: ba4de88f2c4 ll-merge: make callers responsible for showing warnings
4: de8e8f88fa4 = 4: d7a1f4e1f9f merge-ort: capture and print ll-merge warnings in our preferred fashion
5: 6b535a4d55a = 5: cbde4e5d372 merge-ort: mark a few more conflict messages as omittable
6: e2441608c63 = 6: d3e4242a5bd merge-ort: format messages slightly different for use in headers
7: 62734beb693 = 7: 4d79da6e20a diff: add ability to insert additional headers for paths
8: 17eccf7e0d6 = 8: ff9c14b0b7c show, log: include conflict/warning messages in --remerge-diff headers
9: b3e7656cfc6 = 9: aa63860cd0f merge-ort: mark conflict/warning messages from inner merges as omittable
10: ea5df61cf35 = 10: 59d12f213b2 diff-merges: avoid history simplifications when diffing merges
--
gitgitgadget
