[PATCH v2 0/3] kunit: Fix reporting of the skipped parameterized tests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Due to the lack of the SKIP directive in the output, if any of the
parameterized test was skipped, the parser could not recognize that
correctly and was marking the test as PASSED.

This can easily be seen by running the new subtest from patch 1:

$ ./tools/testing/kunit/kunit.py run \
	--kunitconfig ./lib/kunit/.kunitconfig *.example_params*

  [ ] Starting KUnit Kernel (1/1)...
  [ ] ============================================================
  [ ] =================== example (1 subtest) ====================
  [ ] =================== example_params_test  ===================
  [ ] [PASSED] example value 2
  [ ] [PASSED] example value 1
  [ ] [PASSED] example value 0
  [ ] =============== [PASSED] example_params_test ===============
  [ ] ===================== [PASSED] example =====================
  [ ] ============================================================
  [ ] Testing complete. Ran 3 tests: passed: 3

$ ./tools/testing/kunit/kunit.py run \
	--kunitconfig ./lib/kunit/.kunitconfig *.example_params* \
	--raw_output

  [ ] Starting KUnit Kernel (1/1)...
  KTAP version 1
  1..1
      # example: initializing suite
      KTAP version 1
      # Subtest: example
      1..1
          KTAP version 1
          # Subtest: example_params_test
      # example_params_test: initializing
          ok 1 example value 2
      # example_params_test: initializing
          ok 2 example value 1
      # example_params_test: initializing
          ok 3 example value 0
      # example_params_test: pass:2 fail:0 skip:1 total:3
      ok 1 example_params_test
  # Totals: pass:2 fail:0 skip:1 total:3
  ok 1 example

After adding the SKIP directive, the report looks as expected:

  [ ] Starting KUnit Kernel (1/1)...
  [ ] ============================================================
  [ ] =================== example (1 subtest) ====================
  [ ] =================== example_params_test  ===================
  [ ] [PASSED] example value 2
  [ ] [PASSED] example value 1
  [ ] [SKIPPED] example value 0
  [ ] =============== [PASSED] example_params_test ===============
  [ ] ===================== [PASSED] example =====================
  [ ] ============================================================
  [ ] Testing complete. Ran 3 tests: passed: 2, skipped: 1

  [ ] Starting KUnit Kernel (1/1)...
  KTAP version 1
  1..1
      # example: initializing suite
      KTAP version 1
      # Subtest: example
      1..1
          KTAP version 1
          # Subtest: example_params_test
      # example_params_test: initializing
          ok 1 example value 2
      # example_params_test: initializing
          ok 2 example value 1
      # example_params_test: initializing
          ok 3 example value 0 # SKIP unsupported param value
      # example_params_test: pass:2 fail:0 skip:1 total:3
      ok 1 example_params_test
  # Totals: pass:2 fail:0 skip:1 total:3
  ok 1 example

v2: better align with future support for arbitrary levels of testing

Cc: David Gow <davidgow@xxxxxxxxxx>
Cc: Rae Moar <rmoar@xxxxxxxxxx>

Michal Wajdeczko (3):
  kunit/test: Add example test showing parameterized testing
  kunit: Fix reporting of the skipped parameterized tests
  kunit: Update kunit_print_ok_not_ok function

 include/kunit/test.h           |  1 +
 lib/kunit/kunit-example-test.c | 34 +++++++++++++++++++++++++++
 lib/kunit/test.c               | 43 ++++++++++++++++++++++------------
 3 files changed, 63 insertions(+), 15 deletions(-)

-- 
2.25.1




[Index of Archives]     [Linux Wireless]     [Linux Kernel]     [ATH6KL]     [Linux Bluetooth]     [Linux Netdev]     [Kernel Newbies]     [Share Photos]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Samba]     [Device Mapper]

  Powered by Linux