On Thu, May 16, 2024 at 03:19:30AM -0400, Jeff King wrote: > On Sat, May 11, 2024 at 07:21:28PM +0200, Patrick Steinhardt wrote: > > > On Fri, May 10, 2024 at 03:47:39PM -0700, Junio C Hamano wrote: > > > Jeff King <peff@xxxxxxxx> writes: > > [snip] > > > > [1] Another quirk is that we run the whole test suite for both > > > > compilers, which is probably overkill. The main value in comparing > > > > gcc vs clang is that we don't use any constructs that the compiler > > > > complains about. It's _possible_ for there to be a construct that > > > > the compiler does not notice but which causes a runtime difference > > > > (say, undefined behavior which happens to work out on one compiler), > > > > but I think we're again hitting diminishing returns. > > > > > > Yeah, that is a very good point. > > > > On Linux, we have the "pedantic" job that runs Fedora and only compiles > > the sources with DEVOPTS=pedantic without running any of the tests. We > > could do the same on macOS. > > Yeah, I think the infrastructure is there (looks like just resetting > $run_tests). We probably could stand to use it in more places. E.g., is > there even value in running the tests for linux-gcc and linux-clang? > It's _possible_ for there to be a run-time difference in the compiler > outputs, but we may be hitting diminishing returns. The main value I > think is just seeing what the compilers complain about. Actually, scratch that. I forgot we dropped linux-clang last summer in d88d727143 (ci: drop linux-clang job, 2023-06-01), since it was mostly redundant with the sanitizer builds (which use clang). There is still "linux-gcc" and "linux-gcc-default", the former of which uses an older version of the compiler. And again, it's not clear to me that running the actual tests is uncovering useful stuff there. Certainly it is possible, but I wonder if that is the best bang for the buck (e.g., if we wanted to spend a "make test" worth of CPU, then something like osx-sha256 would IMHO be more likely to uncover something useful). -Peff
