On Wed, 31 Jan 2024 15:27:34 -0500 Willem de Bruijn wrote: > Jakub Kicinski wrote: > > +1 I also think we should run and ignore failure. I was wondering if we > > can swap FAIL for XFAIL in those cases: > > > > tools/testing/selftests/kselftest.h > > #define KSFT_XFAIL 2 > > > > Documentation/dev-tools/ktap.rst > > - "XFAIL", which indicates that a test is expected to fail. This > > is similar to "TODO", above, and is used by some kselftest tests. > > > > IDK if that's a stretch or not. Or we can just return PASS with > > a comment? > > Flaky tests will then report both pass and expected fail. That might > add noise to https://netdev.bots.linux.dev/flakes.html? > > I initially considered returning skipped on timing failure. But that > has the same issue. > > So perhaps just return pass? > > > Especially for flaky tests sometimes returning pass and sometimes > returning expected to fa red/green > dash such as Right, we only have pass / fail / skip. (I put the "warn" result in for tests migrated from patchwork so ignore its existence for tests.) We already treat XFAIL in KTAP as "pass". TCP-AO's key-managemeent_ipv6 test for example already reports XFAIL: # ok 15 # XFAIL listen() after current/rnext keys set: the socket has current/rn ext keys: 100:200 Skips look somewhat similar in KTAP, "ok $number # SKIP" but we fish those out specifically to catch skips. Any other "ok .... # comment" KTAP result is treated as a "pass" right now.
