Re: [PATCH 2/3] ci: avoid bare "gcc" for osx-gcc job

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]


On Fri, May 17, 2024 at 09:59:35AM -0700, Junio C Hamano wrote:

> Jeff King <peff@xxxxxxxx> writes:
> > Yeah. Combining orthogonal properties into a single job lets us cover
> > both (for the common case of success on both) with less CPU. But:
> >
> >   - it can sometimes be hard to figure out which of the properties was
> >     responsible for a failure. That was the very subject of the thread I
> >     referenced earlier, where "linux-gcc" was "use gcc" and also "set
> >     lots of knobs".
> >
> >   - they might not actually be orthogonal. If you care about checking
> >     runtime behavior in the output of two compilers, then that _could_
> >     manifest only in the sha256 code. Or as you get into more
> >     properties, they may overlap in other ways. I think reftable+sha256
> >     is an interesting (eventual) combo to test on top of reftable+sha1.
> We could consider permuting, then?  If we (for the sake of
> simplicity) had two jobs available, one compiled with GCC and the
> other compiled with clang, we can enumerate other properties
> (e.g. <SHA-1 vs SHA-256>, <reftable vs reffiles>) into pairs, and in
> one run, GCC may be running SHA-1+reffiles while clang is running
> SHA-256+reftable, and in another run, GCC may be running
> SHA-256+reffiles, etc. --- eventually we cover all four combinations
> (admittedly for different commits).

That's a neat idea to get eventual coverage. I have a feeling it would
be a pain in practice, though, because now the CI results aren't quite
deterministic. So if commit X introduces a bug in some combination, we
might not find out until later, and seeing that X passed all tests
doesn't absolve it of responsibility anymore.

Likewise, I often have to re-run CI to get more data, or to see if a
failure is a flake. If it changed what it ran that would be confusing
(though I guess you could use the commit hash as the random "seed" for
deciding which permutation to run).


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux