On 15/12/2023 13:54, Mark Brown wrote: > On Thu, Dec 14, 2023 at 04:24:34PM +0000, Ryan Roberts wrote: >> When running tests on a CI system (e.g. LAVA) it is useful to output >> test results in TAP format so that the CI can parse the fine-grained >> results to show regressions. Many of the mm selftest binaries already >> output using the TAP format. And the kselftests runner >> (run_kselftest.sh) also uses the format. CI systems such as LAVA can >> already handle nested TAP reports. However, with the mm selftests we >> have 3 levels of nesting (run_kselftest.sh -> run_vmtests.sh -> >> individual test binaries) and the middle level did not previously >> support TAP, which breaks the parser. > > Reviewed-by: Mark Brown <broonie@xxxxxxxxxx> Thanks! > >> Let's fix that by teaching run_vmtests.sh to output using the TAP >> format. Ideally this would be opt-in via a command line argument to >> avoid the possibility of breaking anyone's existing scripts that might >> scrape the output. However, it is not possible to pass arguments to >> tests invoked via run_kselftest.sh. So I've implemented an opt-out >> option (-n), which will revert to the existing output format. > > What I did for ftrace which had a similar situation was make a wrapper > script which invokes the test runner, make the test runner a > TEST_PROGS_EXTENDED so it's not run by the kselftest infrastructure > automatically and make the wrapper a normal TEST_PROGS. Neither option > is especially lovely. Yeah that's a good idea... I'll wait and see if anyone shouts that this has broken something. If nothing is broken, I think it is better to just make TAP the default rather than adding yet another wrapper.
