On 13/07/2023 16:43, Mark Brown wrote: > On Thu, Jul 13, 2023 at 04:36:18PM +0100, Ryan Roberts wrote: >> On 13/07/2023 16:30, Mark Brown wrote: > >>> The results parsers I'm aware of like the LAVA one will DTRT with nested >>> kselftests since that's required to pull see individual test cases run >>> by a single binary so it's the common case to see at least one level of >>> nesting. > >> That's good to hear. But bear in mind that run_vmtests.sh does not use TAP. So >> you end up with a single top-level test who's result is reported with >> run_kselftest.sh's TAP output. Then you have a second level (run_vmtests.sh) >> using custom reporting, then _some_ of the tests invoked use TAP so you >> sometimes have TAP at level 3. But those tests at level 2 that don't do their >> own TAP output probably won't be parsed by LAVA? > > I think that should mostly mean that all the tests that don't > individually produce KTAP output get ignored by parsers and those which > do produce KTAP output will be seen as nesting one level up from where > they are (ie, the individual cases will run directly from vmtest), > though there's likely to be confusion about expected run numbers for > things that actually pay attention to that. I suspect it wouldn't be technically dififcult to add a --tap option to run_vmtests.sh, which would switch the output format to TAP. If people are amenable. > >> Since you agreed to put this into the CI, I was going to call this part "your >> problem" ;-) > > It'll run, the results are a different story. :P
