Re: [PATCH v3 1/3] selftests/tc-testing: add exit code

Jamal Hadi Salim <jhs@xxxxxxxxxxxx> · Fri, 3 Dec 2021 07:40:58 -0500

All LGTM - but shouldnt these patches be independent of each other?
i.e submit 3 patches each with 1/1

cheers,
jamal

On 2021-12-02 21:53, Li Zhijian wrote:
Mark the summary result as FAIL to prevent from confusing the selftest
framework if some of them are failed.

Previously, the selftest framework always treats it as *ok* even though
some of them are failed actually. That's because the script tdc.sh always
return 0.

  # All test results:
  #
  # 1..97
  # ok 1 83be - Create FQ-PIE with invalid number of flows
  # ok 2 8b6e - Create RED with no flags
[...snip]
  # ok 6 5f15 - Create RED with flags ECN, harddrop
  # ok 7 53e8 - Create RED with flags ECN, nodrop
  # ok 8 d091 - Fail to create RED with only nodrop flag
  # ok 9 af8e - Create RED with flags ECN, nodrop, harddrop
  # not ok 10 ce7d - Add mq Qdisc to multi-queue device (4 queues)
  #       Could not match regex pattern. Verify command output:
  # qdisc mq 1: root
  # qdisc fq_codel 0: parent 1:4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
  # qdisc fq_codel 0: parent 1:3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
[...snip]
  # ok 96 6979 - Change quantum of a strict ETS band
  # ok 97 9a7d - Change ETS strict band without quantum
  #
  #
  #
  #
  ok 1 selftests: tc-testing: tdc.sh <<< summary result

CC: Philip Li <philip.li@xxxxxxxxx>
Reported-by: kernel test robot <lkp@xxxxxxxxx>
Signed-off-by: Li Zhijian <zhijianx.li@xxxxxxxxx>
Acked-by: Davide Caratti <dcaratti@xxxxxxxxxx>
Acked-by: Jamal Hadi Salim <jhs@xxxxxxxxxxxx>
---
V3: repost to netdev
V2: Fix missing ':'
---
  tools/testing/selftests/tc-testing/tdc.py | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py
index a3e43189d940..ee22e3447ec7 100755
--- a/tools/testing/selftests/tc-testing/tdc.py
+++ b/tools/testing/selftests/tc-testing/tdc.py
@@ -716,6 +716,7 @@ def set_operation_mode(pm, parser, args, remaining):
          list_test_cases(alltests)
          exit(0)
  
+    exit_code = 0 # KSFT_PASS
      if len(alltests):
          req_plugins = pm.get_required_plugins(alltests)
          try:
@@ -724,6 +725,8 @@ def set_operation_mode(pm, parser, args, remaining):
              print('The following plugins were not found:')
              print('{}'.format(pde.missing_pg))
          catresults = test_runner(pm, args, alltests)
+        if catresults.count_failures() != 0:
+            exit_code = 1 # KSFT_FAIL
          if args.format == 'none':
              print('Test results output suppression requested\n')
          else:
@@ -748,6 +751,8 @@ def set_operation_mode(pm, parser, args, remaining):
                          gid=int(os.getenv('SUDO_GID')))
      else:
          print('No tests found\n')
+        exit_code = 4 # KSFT_SKIP
+    exit(exit_code)
  
  def main():
      """
@@ -767,8 +772,5 @@ def main():
  
      set_operation_mode(pm, parser, args, remaining)
  
-    exit(0)
-
-
  if __name__ == "__main__":
      main()








