On 22/02/19 16:30, Stefan Raspl wrote: > On 22.02.19 15:38, Stefan Raspl wrote: >> The current tap13 support summarizes testcases on a rather coarse >> granularity. Which sort of defeats the purpose, especially in CI >> environments, where having results for individual tests to pinpoint >> problems helps a lot. Plus it can be convenient to have the plan in front >> of the individual testcases. >> This patch improves things by post-processing the log files, extracting >> results for individual testcases, introduces test case numbers and puts the >> plan in front of the test case results. >> One could argue that post-processing the logs is a rather fragile approach. >> Which is true - but apparently already the case, see e.g. extract_summary >> in scripts/runtime.bash. Plus this is quite cheap, while a proper solution >> would require to modify the kernels, so we can e.g. pass in a counter for >> the test case number. And we would probably have to come up with reasonable >> test case names, while this approach simply derives them from each test >> output. >> >> Before: >> $ ./run_tests.sh -t >> TAP version 13 >> ok selftest-setup >> ok intercept >> ok emulator >> ok sieve >> ok sthyi >> ok skey >> ok diag10 >> ok pfmf >> ok cmm >> ok vector >> ok gs # SKIP >> ok iep # SKIP >> 1..12 >> >> After: >> $ ./run_tests.sh -t >> TAP version 13 >> 1..180 > > Minor correction: With the latest changes, the plan actually appears after the > individual testcases at the end, not as the 2nd line. I sent another version that does not need a lock with a very low tech solution: just correct the numbers as a postprocessing pass. :) While I was at it I also moved all TAP handling to run_tests.sh. Paolo
