On Fri, 9 Aug 2024 21:19:11 -0700, Jakub Kicinski wrote: > > On Thu, 8 Aug 2024 09:23:09 -0700, Jakub Kicinski wrote: > > > A number of checks now return SKIP because veth doesn't support all > > > ethtool APIs. > > > > > > In netdev selftests we try to make sure SKIP is only used when test > > > cannot be performed because of limitations of the environment. > > > For example some tool is not installed, kernel doesn't have a config. > > > Something that the person running the test is able to fix by fixing > > > how the test is run. > > > > > > Running this test on veth will always SKIP, nothing CI system can do. > > > Please make the test use the keyword XFAIL instead of SKIP when > > > functionality is not supported by the underlying driver. > > > > Ack, understood. I will do that, one clarification though. > > Currently, the tests are using either PASS or FAIL and no SKIP. Based on > > the above suggestion, it seems that I have replace FAIL with XFAIL for all > > the tests that fail due to functionality not being supported by the > > underlying driver. > > Right, sorry for lack of clarity. > > Our CI doesn't fully trust the exit codes, so even though the test > exits with zero the CI parses the output and finds the "SKIP: ..." > lines. You need to replace those "SKIP"s in the output with "XFAIL". I re-tested and found that currently only two APIs are tested, "dump" and "stats". For veth pair, the only test that fails currently with a SKIP is the dump operation. ``` # Cannot get register dump: Operation not supported # SKIP: veth1: ethtool dump not supported ``` This is present in kci_netdev_ethtool_test function, please confirm if this is the one that I need to change to XFAIL. The logic to incorporate the failure exit code for dump operation (74) is already in place. ``` kci_netdev_ethtool_test 74 'dump' "ethtool -d $netdev" ``` I just need to change the SKIP in this function to XFAIL. In case, if you were referring to any other tests that are failing (features on/off), please let me know. Thank you for prompt feedbacks, really appreciated.
