Junio C Hamano wrote:
> Todd Zullinger <tmz@xxxxxxxxx> writes:
>
>> After 1f010d6bdf (doc: use .adoc extension for AsciiDoc files,
>> 2025-01-20), we no longer matched any files in this test. The result is
>> that we did not test for mismatches in the documentation and --help
>> output.
>>
>> Adjust the test to look at the renamed *.adoc files.
>>
>> Signed-off-by: Todd Zullinger <tmz@xxxxxxxxx>
>> ---
>> t/t0450-txt-doc-vs-help.sh | 50 +++++++++----------
>> ...t-help-mismatches => adoc-help-mismatches} | 0
>> 2 files changed, 25 insertions(+), 25 deletions(-)
>> rename t/t0450/{txt-help-mismatches => adoc-help-mismatches} (100%)
>
> Wow, good find. This is especially bad and I am glad we caught it
> before -rc1 (the hope was that all of these should have been caught
> while the topic was in 'next', which was the whole point of cooking
> it longer in 'next' than usual, but that plan did not really work).
Indeed. I haven't looked closely at the CI bits to see how
we might be able to improve this. When we skip tests
unintentionally we just don't see that currently.
I have a file of patterns I expected to skip in the rpm
builds when I maintained git in Fedora. I'd regularly run:
grep -E '# SKIP|skipped:' build.log | grep -Evf git.skip-test-patterns
to watch for skipped tests[1]. Those often indicated that I
needed to add a new build dependency, but sometimes catch
issues like this. We have some similar things in the CI
scripts, I think. Maybe we could add one for this sort of
thing.
What I don't know is how many tests we skip now and how
those differ across the multiple test runs. It might be
annoyingly variable to track what is skipped between the
multiple OS types, releases, and build options.
I don't even know if we have the same data in the build
logs. The rpm builds I run use `prove` with verbose output.
[1] That is also how I noticed that we've been skipping all
tests which rely on the GPG2 prereq, as I mentioned
recently in <Z8HVkqqD054QGPIE@xxxxxxxxxxxxxxx>.
--
Todd
