The current selftests infrastructure formats the results in TAP 13. This
version doesn't support subtests and only the end result of each
selftest is taken into account. It means that a single issue in a
subtest of a selftest containing multiple subtests forces the whole
selftest to be marked as failed. It also means that subtests results are
not tracked by CI executing selftests.
MPTCP selftests run hundreds of various subtests. It is then important
to track each of them and not one result per selftest.
It is particularly interesting to do that when validating stable kernels
with the last version of the test suite: tests might fail because a
feature is not supported but the test didn't skip that part. In this
case, if subtests are not tracked, the whole selftest will be marked as
failed making the other subtests useless because their results are
ignored.
Regarding this patch set:
- The two first patches modify connect and userspace_pm selftests to
continue executing other tests if there is an error before the end.
This is what is done in the other MPTCP selftests.
- Patches 3-5 are refactoring the code in userspace_pm selftest to
reduce duplicated code, suppress some shellcheck warnings and prepare
subtests' support by using new helpers.
- Patch 6 adds new helpers in mptcp_lib.sh to easily support printing
the subtests results in the different MPTCP selftests.
- Patch 7-13 format subtests results in TAP 13 in the different MPTCP
selftests.
Signed-off-by: Matthieu Baerts <matthieu.baerts@xxxxxxxxxxxx>
---
Matthieu Baerts (13):
selftests: mptcp: connect: don't stop if error
selftests: mptcp: userspace pm: don't stop if error
selftests: mptcp: userspace_pm: fix shellcheck warnings
selftests: mptcp: userspace_pm: uniform results printing
selftests: mptcp: userspace_pm: reduce dup code around printf
selftests: mptcp: lib: format subtests results in TAP
selftests: mptcp: connect: format subtests results in TAP
selftests: mptcp: pm_netlink: format subtests results in TAP
selftests: mptcp: join: format subtests results in TAP
selftests: mptcp: diag: format subtests results in TAP
selftests: mptcp: simult flows: format subtests results in TAP
selftests: mptcp: sockopt: format subtests results in TAP
selftests: mptcp: userspace_pm: format subtests results in TAP
tools/testing/selftests/net/mptcp/diag.sh | 7 +
tools/testing/selftests/net/mptcp/mptcp_connect.sh | 66 ++++++--
tools/testing/selftests/net/mptcp/mptcp_join.sh | 37 ++++-
tools/testing/selftests/net/mptcp/mptcp_lib.sh | 66 ++++++++
tools/testing/selftests/net/mptcp/mptcp_sockopt.sh | 20 ++-
tools/testing/selftests/net/mptcp/pm_netlink.sh | 6 +-
tools/testing/selftests/net/mptcp/simult_flows.sh | 4 +
tools/testing/selftests/net/mptcp/userspace_pm.sh | 181 +++++++++++++--------
8 files changed, 298 insertions(+), 89 deletions(-)
---
base-commit: 60cc1f7d0605598b47ee3c0c2b4b6fbd4da50a06
change-id: 20230712-upstream-net-next-20230712-selftests-mptcp-subtests-25d250d77886
Best regards,
--
Matthieu Baerts <matthieu.baerts@xxxxxxxxxxxx>
