On Tue, Jun 16, 2020 at 12:44:28PM -0700, Brendan Higgins wrote: > On Tue, Jun 16, 2020 at 9:42 AM Bird, Tim <Tim.Bird@xxxxxxxx> wrote: > > > From: Paolo Bonzini <pbonzini@xxxxxxxxxx> > > > > > > On 15/06/20 21:07, Bird, Tim wrote: > > > >> Note: making the plan line required differs from TAP13 and TAP14. I > > > >> think it's the right choice, but we should be clear. > > > > > > As an aside, where is TAP14? > > By TAP14, I was referring to the current, undocumented, KUnit > > conventions. > > Not so. TAP14 is the proposed next version of TAP13: > > https://github.com/TestAnything/testanything.github.io/pull/36 > https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md I was reading this (I haven't compared to the blob above): https://github.com/TestAnything/Specification/blob/tap-14-specification/specification.md > Based on the discussion, it seems like most of the things we wanted > from TAP14 would probably make it in if TAP ever accepts another pull > request. Were our leading diagnostic lines part of their latest spec? I thought we were pretty far off in left field for that particular bit. > > My personal preference is to have the dash. I think it's more human readable. > > I note that the TAP spec has examples of result lines both with and without > > the dash, so even the spec is ambiguous on this. I think not mandating it > > either way is probably best. For regex parsers, it's easy to ignore with '[-]?' > > outside the pattern groups that grab the number and description. > > I don't think we care, because we don't use it. Yeah, I'm in the same place. I don't care -- I would just like a determination. (The "implied" nature of it in TAP14 bothers me.) > > > XFAIL/XPASS are different from SKIP. I personally don't have a need for > > > them, but kselftests includes XFAIL/XPASS exit codes and they aren't > > > reflected into selftests/kselftest/runner.sh. > > > > > > Likewise, kselftest.h has ksft_inc_xfail_cnt but not > > > ksft_test_result_xfail/ksft_test_result_xpass. I proposed fixing that recently[1]. seccomp uses XFAIL for "I have detected you lack the config to test this, so I can't say it's working or not, because it only looks like a failure without the config." [1] https://lore.kernel.org/lkml/20200611224028.3275174-7-keescook@xxxxxxxxxxxx/ -- Kees Cook
