Jakub Kicinski wrote: > On Wed, 31 Jan 2024 15:27:34 -0500 Willem de Bruijn wrote: > > Jakub Kicinski wrote: > > > +1 I also think we should run and ignore failure. I was wondering if we > > > can swap FAIL for XFAIL in those cases: > > > > > > tools/testing/selftests/kselftest.h > > > #define KSFT_XFAIL 2 > > > > > > Documentation/dev-tools/ktap.rst > > > - "XFAIL", which indicates that a test is expected to fail. This > > > is similar to "TODO", above, and is used by some kselftest tests. > > > > > > IDK if that's a stretch or not. Or we can just return PASS with > > > a comment? > > > > Flaky tests will then report both pass and expected fail. That might > > add noise to https://netdev.bots.linux.dev/flakes.html? > > > > I initially considered returning skipped on timing failure. But that > > has the same issue. > > > > So perhaps just return pass? > > > > > > Especially for flaky tests sometimes returning pass and sometimes > > returning expected to fa red/green > > dash such as > > Right, we only have pass / fail / skip. (I put the "warn" result in for > tests migrated from patchwork so ignore its existence for tests.) > > We already treat XFAIL in KTAP as "pass". TCP-AO's key-managemeent_ipv6 > test for example already reports XFAIL: Ok perfect. Then I'll do the same. > # ok 15 # XFAIL listen() after current/rnext keys set: the socket has current/rn > ext keys: 100:200 > > Skips look somewhat similar in KTAP, "ok $number # SKIP" but we fish > those out specifically to catch skips. Any other "ok .... # comment" > KTAP result is treated as a "pass" right now.
