On 2020-06-16 18:58, Kees Cook wrote: > On Tue, Jun 16, 2020 at 12:44:28PM -0700, Brendan Higgins wrote: >> On Tue, Jun 16, 2020 at 9:42 AM Bird, Tim <Tim.Bird@xxxxxxxx> wrote: >>>> From: Paolo Bonzini <pbonzini@xxxxxxxxxx> >>>> >>>> On 15/06/20 21:07, Bird, Tim wrote: >>>>>> Note: making the plan line required differs from TAP13 and TAP14. I >>>>>> think it's the right choice, but we should be clear. >>>> >>>> As an aside, where is TAP14? >>> By TAP14, I was referring to the current, undocumented, KUnit >>> conventions. >> >> Not so. TAP14 is the proposed next version of TAP13: >> >> https://github.com/TestAnything/testanything.github.io/pull/36 >> https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md > > I was reading this (I haven't compared to the blob above): > > https://github.com/TestAnything/Specification/blob/tap-14-specification/specification.md > >> Based on the discussion, it seems like most of the things we wanted >> from TAP14 would probably make it in if TAP ever accepts another pull >> request. > > Were our leading diagnostic lines part of their latest spec? I thought > we were pretty far off in left field for that particular bit. > >>> My personal preference is to have the dash. I think it's more human readable. >>> I note that the TAP spec has examples of result lines both with and without >>> the dash, so even the spec is ambiguous on this. I think not mandating it >>> either way is probably best. For regex parsers, it's easy to ignore with '[-]?' >>> outside the pattern groups that grab the number and description. >> >> I don't think we care, because we don't use it. > > Yeah, I'm in the same place. I don't care -- I would just like a > determination. (The "implied" nature of it in TAP14 bothers me.) > >>>> XFAIL/XPASS are different from SKIP. I personally don't have a need for >>>> them, but kselftests includes XFAIL/XPASS exit codes and they aren't >>>> reflected into selftests/kselftest/runner.sh. >>>> >>>> Likewise, kselftest.h has ksft_inc_xfail_cnt but not >>>> ksft_test_result_xfail/ksft_test_result_xpass. > > I proposed fixing that recently[1]. seccomp uses XFAIL for "I have > detected you lack the config to test this, so I can't say it's working > or not, because it only looks like a failure without the config." Based on that description, the case sounds like it should be a skip. Or if the entire test depends on the missing config then Bail out might be appropriate. > > [1] https://lore.kernel.org/lkml/20200611224028.3275174-7-keescook@xxxxxxxxxxxx/ >
