"Philip Oakley" <philipoakley@xxxxxxx> writes:
> For an integrator, or especially a CI tool, simply checking the second
> parents of each topic merge (post fail) should at least indicate if
> the basics of the feature actually passed the tests, though it doesn't
> check for interaction issues. This could give direct author feedback!
I think that is essentially what Avery advocated for in the old
thread Christian gave us earlier in the thread, and I agree that is
a useful way to help contributors.
While we are on the topic of testing efficiently and effectively,
there are a few other things worth taking advantage of.
One obvious thing is that not all topic branches get updated every
day, even though the tip of 'pu' will be different every day (or on
days when I run the integration cycle more than once, after every
pushout), because we constantly acquire new topics or updated ones.
The tips of topics that are merged in 'pu^{/^### match next}..pu'
can be enumerated by following the first-parent chain, and they can
be tested all one-by-one, excluding the ones that have been tested
already in the previous runs. When I counted these merges the other
day I had 27 of them, and as of this writing I have 20 of them
(because some topics were merged to 'next' in the meantime). Among
them, only 3 are new. Everything else would have already been
tested if such a test was done daily. So from that point of view,
testing them all could be less expensive than bisecting; in order to
bisect a first-parent chain of 20 merges, you'd need to test 5 or
so.
Another thing that may help to prevent breakages from seeping into
'next' is that "pu^{/^### match next}..pu" are rebuilt every day,
and the order of topics that are merged are updated. The topics
that are closer to be merged to 'next' are moved down, so that
testing the merge result would give us closer result to what would
testing 'next' in the near future would give us [*1*]. So paying
closer attention to the merges (not tips of topics) above the commit
marked as "pu^{/^### match next}" when testing would catch potential
breakage about to happen in 'next' due to unexpected interactions of
topics when merged together.
There is another point in the history that may be worth paying
closer attention, which is the tip of 'jch'. This always is ahead
of "pu^{/^### match next}", and almost always has a handful more
topics, many of which are considered to be merged to 'next' in near
future. The branch is meant to be at least stable enough for my
personal use (hence its name) in helping me run everyday integration
cycles.
[Footnote]
*1* Suppose there are three topics A, B and C that are not yet in
'next', and 'pu' merges them in that order. Further suppose
that C is a lot better cooked than others. Merging C directly
to 'next' however can expose a hidden issue that changes C
introduces alone is not sufficient and depends on A or B to be
present. To avoid such a surprise after merging C to 'next',
when 'pu' is rebuilt, I try to reorder them so that C is merged
first on top of an equivalent of 'next', and then A and B. For
the same reason, topics in 'master..pu^{/^### match next}' are
also reordered so that ones that are planned to be merged to
'master' soon comes at the bottom.
One natural consequence of this is that 'pu' is rebuilt directly
on top of 'master' and 'next' does not fast-forwared to 'pu'.
This arrangement also helps to spot mismerges to 'next' and
avoid the same mismerge to affect 'master' when topics are
merged to it.