On Fri, Dec 13, 2024 at 10:58:47AM +0100, Toon Claes wrote:
> Patrick Steinhardt <ps@xxxxxx> writes:
> > diff --git a/t/meson.build b/t/meson.build
> > index 9e676e69363ed6311426500d98fe281e30d26bcb..f1fbc6ae179079f4d5d86f9a60956fad84d0495c 100644
> > --- a/t/meson.build
> > +++ b/t/meson.build
> > @@ -1092,6 +1092,42 @@ integration_tests = [
> > 't9903-bash-prompt.sh',
> > ]
> >
> > +# Sanity check that we are not missing any tests present in 't/'. This check
> > +# only runs once at configure time and is thus best-effort, only. It is
> > +# sufficient to catch missing test suites in our CI though.
> > +foreach glob, tests : {
> > + 't[0-9][0-9][0-9][0-9]-*.sh': integration_tests,
> > + 'unit-tests/t-*.c': unit_test_programs,
> > + 'unit-tests/u-*.c': clar_test_suites,
> > +}
> > + actual_tests = run_command(shell, '-c', 'ls ' + glob,
> > + check: true,
> > + env: script_environment,
> > + ).stdout().strip().split('\n')
> > +
> > + if tests != actual_tests
> > + missing_tests = [ ]
> > + foreach actual_test : actual_tests
> > + if actual_test not in tests
> > + missing_tests += actual_test
> > + endif
> > + endforeach
> > + if missing_tests.length() > 0
> > + error('Missing tests:\n\n - ' + '\n - '.join(missing_tests))
>
> This gives nice output:
>
> $ touch t/unit-tests/u-bar.c t/unit-tests/u-foo.c
>
> $ meson setup builddir --reconfigure
>
> The Meson build system
> Version: 1.4.1
> [snip]
> Configuring update.sample using configuration
> Configuring exclude using configuration
>
> t/meson.build:1116:6: ERROR: Problem encountered: Missing tests:
>
> - unit-tests/u-bar.c
> - unit-tests/u-foo.c
>
> A full log can be found at git/builddir/meson-logs/meson-log.txt
>
> But I think the error message is a little bit confusing. It sounds like
> the test file is missing, but it's the configuration of the test that is
> missing.
>
> Now I've realized it hard to write a good error message here. But I
> would suggest something like "Tests files found, but not configured".
>
> > + endif
> > +
> > + superfluous_tests = [ ]
> > + foreach integration_test : tests
> > + if integration_test not in actual_tests
> > + superfluous_tests += integration_test
> > + endif
> > + endforeach
> > + if superfluous_tests.length() > 0
> > + error('Superfluous tests:\n\n - ' + '\n - '.join(superfluous_tests))
>
> Also here I would suggest different error message, maybe something like
> "Tests configured, but file not found"
Both good suggestions, thanks!
Patrick
