On 19/12/2023 00:55, John Hubbard wrote: > On 12/18/23 16:51, John Hubbard wrote: >> On 12/18/23 03:32, Ryan Roberts wrote: >> ... >>>> I should also point out that some of the subtests already attempt a TAP >>>> output. So now we end up with TAP-within-TAP output for those programs. >>> >>> It's actually TAP-in-TAP-in-TAP if you're running from run_kselftest.sh :) >>> >>>> >>>> For example: >>>> # ----------------------- >>>> # running ./madv_populate >>>> # ----------------------- >>>> # TAP version 13 >>>> # 1..21 >>>> # # [RUN] test_prot_read >>>> # ok 1 MADV_POPULATE_READ with PROT_READ >>>> # ok 2 MADV_POPULATE_WRITE with PROT_READ >>>> # # [RUN] test_prot_write >>>> # ok 3 MADV_POPULATE_READ with PROT_WRITE >>>> ...etc... >>>> >>>> Note the double level of leading '#' characters. >>>> >>>> Again, this is still readable enough for humans. But it should probably >>>> be removed in subsequent patches to the subtests. >>> >>> I personally don't agree with this. It would be difficult to flatten to a single >>> TAP instance because the top level doesn't have a clue how many test cases the >> >> That's not quite what I had in mind... >> >>> child is running. Trying to do this will make things more fragile and less >>> modular. LAVA can certainly deal with nested test cases and correctly parses >>> everything to test case names that contain the test name at each level of >>> nesting. The thing I was trying to solve with this patch was that previously the >>> top level (run_kselftest.sh) and the bottom level (individual mm test binaries) >>> were using TAP, but the middle level (run_vmtests.sh) wasn't, and this was >>> confusing the LAVA parser. >>> >> >> I was thinking more along these lines: >> >> a) For the individual programs (binaries), there is actually neither need nor >> desire to create TAP output at that level, because frameworks like LAVA only >> care about running a lot of tests and parsing the output. >> >> b) Therefore, just stop specifying TAP output at the leaf level, and let >> run_vmtests.sh and run_kselftest.sh do it. >> >> Looking at madv_populate.c, I see that it scatters calls to ksft_*() around. >> And I was thinking that this is all just redundant, isn't it? >> > > Although I suppose that the counter argument is that the subtests in > madv_populate.c really *do* want to be specifically printed in TAP > format. > > arggh, I guess this is just not worth fooling around with after all. Yes; I wouldn't want to lose the fine granularity we have currently. For example cow.c has ~900 test cases now that I've multiplied everything up for mTHP. 16 of those are known to fail (hugetlb issue) and 1 is skipped. I wouldn't want to reduce that down to a single cow test case that always fails; that's not helpful to understand if I've regressed something. But sounds like we are both on the same page now. > > > thanks,
