Hi Rae, On Thu, Mar 06, 2025 at 11:02:13AM -0500, Rae Moar wrote: > On Thu, Mar 6, 2025 at 7:26 AM Brendan Jackman <jackmanb@xxxxxxxxxx> wrote: > > Since I reported this bug, I discovered that the example above is in > > fact valid TAP: > > > > > The plan [...] must appear once, whether at the beginning or end of the output. > > > > From https://testanything.org/tap-version-13-specification.html > > Hi! > This brings up an interesting question because the parser has been > mainly geared towards parsing KTAP > (https://docs.kernel.org/dev-tools/ktap.html) rather than TAP. > (Although we do try to have backwards compatibility with TAP v14 > "Subtest" lines) > > For example, > > TAP version 13 > 1..1 > TAP version 13 > 1..1 > ok 1 test_case > ok 1 test_suite > > This would be accepted by the parser without error because it is valid > KTAP even though it is not valid TAP v13. > > The scenario above that caused the infinite loop would be incorrect > KTAP (which requires the test plan to follow a version line) but > correct TAP v13. So do we accept it without error? Ideally, we would > parse based on the version given in the version line. > > Just an interesting thought. Either way, I will remove the error for > now as our parameterized tests don't properly produce a test plan, > which causes errors. OK yeah sounds good. Now I think about it, I should note that I am abusing the KUnit scripts to parse the result of a kselftest run. I just assumed that this would be supported but actually there's no particular reason I should have assumed that :) I would like to have general support in the tree for parsing test output ([K]TAP it's not really a human-readable format IMO... even in the best case like nice tidy KUnit tests I find the structure very hard to read. And without something like 'kunit.py parse' I find it extremely difficult to get a high-level gestalt of how a test run went). But that doesn't mean kunit.py is responsible for the whole kernel tree's output! Still, it would be nice to handle it to the extent that's practical (and at least, with no infinite loops :D).
