On 6/12/24 1:37 AM, Shuah Khan wrote:
> On 6/9/24 23:41, Muhammad Usama Anjum wrote:
>> Conform individual tests to TAP output. One patch conform one test. With
>> this series, all vDSO tests become TAP conformant.
>>
>> First patch conform the test by using kselftest_harness.h. Other patches
>> are conforming using default kselftest.h helpers.
>>
>> All tests have been tested multiple times before and after these
>> patches. They are working correctly and outputting TAP messaging to find
>> failures quikly when they happen.
>> ---
>> Changes since v1:
>> - Update cover letter
>> - Update commit message of first patch
>>
>> Muhammad Usama Anjum (4):
>> kselftests: vdso: vdso_test_clock_getres: conform test to TAP output
>> kselftests: vdso: vdso_test_correctness: conform test to TAP output
>> kselftests: vdso: vdso_test_getcpu: conform test to TAP output
>> kselftests: vdso: vdso_test_gettimeofday: conform test to TAP output
>>
>> .../selftests/vDSO/vdso_test_clock_getres.c | 68 ++++----
>> .../selftests/vDSO/vdso_test_correctness.c | 146 +++++++++---------
>> .../testing/selftests/vDSO/vdso_test_getcpu.c | 16 +-
>> .../selftests/vDSO/vdso_test_gettimeofday.c | 23 +--
>> 4 files changed, 126 insertions(+), 127 deletions(-)
>>
>
> I see two changes:
>
> - concvering this test to use kselftes_harness. I am not seeing a value to
> this.
> - Second converting every single print to TAP. This isn't necessary since the
> kselftest wrapper prints appropriate summary message based on the KSFT_ code
If a test prints standard output, only then CI and regression detection
scripts identify if a test (and test case) was passed or failed. Or when it
started failing. The attempt here to conform to TAP to get output which is
recognizable and can easily be processed.
>
> I want to output before and after to assess the value of this change. TAP
> conversions
> make sense if and when they add value.
Sure. Attaching before and after output for easy comparison. Please let me
know about your thoughts.
>
> thanks,
> -- Shuah
>
--
BR,
Muhammad Usama Anjum
TAP version 13
1..6
# timeout set to 45
# selftests: vDSO: vdso_test_gettimeofday
# TAP version 13
# 1..1
# ok 1 The time is 1718178343.212990
# # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 1 selftests: vDSO: vdso_test_gettimeofday
# timeout set to 45
# selftests: vDSO: vdso_test_getcpu
# TAP version 13
# 1..1
# ok 1 Running on CPU 12 node 0
# # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 2 selftests: vDSO: vdso_test_getcpu
# timeout set to 45
# selftests: vDSO: vdso_test_abi
# TAP version 13
# 1..16
# # [vDSO kselftest] VDSO_VERSION: LINUX_2.6
# # The time is 1718178343.302971
# ok 1 __vdso_gettimeofday
# # clock_id: CLOCK_REALTIME
# # The time is 1718178343.302984142
# ok 2 __vdso_clock_gettime CLOCK_REALTIME
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 3 __vdso_clock_getres CLOCK_REALTIME
# # clock_id: CLOCK_BOOTTIME
# # The time is 1373292.867443353
# ok 4 __vdso_clock_gettime CLOCK_BOOTTIME
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 5 __vdso_clock_getres CLOCK_BOOTTIME
# # clock_id: CLOCK_TAI
# # The time is 1718178343.303018993
# ok 6 __vdso_clock_gettime CLOCK_TAI
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 7 __vdso_clock_getres CLOCK_TAI
# # clock_id: CLOCK_REALTIME_COARSE
# # The time is 1718178343.299307907
# ok 8 __vdso_clock_gettime CLOCK_REALTIME_COARSE
# # The vdso resolution is 0 4000000
# # The syscall resolution is 0 4000000
# ok 9 __vdso_clock_getres CLOCK_REALTIME_COARSE
# # clock_id: CLOCK_MONOTONIC
# # The time is 399550.224742318
# ok 10 __vdso_clock_gettime CLOCK_MONOTONIC
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 11 __vdso_clock_getres CLOCK_MONOTONIC
# # clock_id: CLOCK_MONOTONIC_RAW
# # The time is 399550.182218615
# ok 12 __vdso_clock_gettime CLOCK_MONOTONIC_RAW
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 13 __vdso_clock_getres CLOCK_MONOTONIC_RAW
# # clock_id: CLOCK_MONOTONIC_COARSE
# # The time is 399550.221001200
# ok 14 __vdso_clock_gettime CLOCK_MONOTONIC_COARSE
# # The vdso resolution is 0 4000000
# # The syscall resolution is 0 4000000
# ok 15 __vdso_clock_getres CLOCK_MONOTONIC_COARSE
# # The time in hours since January 1, 1970 is 477271
# ok 16 __vdso_time
# # Totals: pass:16 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 3 selftests: vDSO: vdso_test_abi
# timeout set to 45
# selftests: vDSO: vdso_test_clock_getres
# TAP version 13
# 1..7
# # Starting 7 tests from 1 test cases.
# # RUN global.clock_realtime ...
# # OK global.clock_realtime
# ok 1 global.clock_realtime
# # RUN global.clock_boottime ...
# # OK global.clock_boottime
# ok 2 global.clock_boottime
# # RUN global.clock_tai ...
# # OK global.clock_tai
# ok 3 global.clock_tai
# # RUN global.clock_realtime_coarse ...
# # OK global.clock_realtime_coarse
# ok 4 global.clock_realtime_coarse
# # RUN global.clock_monotonic ...
# # OK global.clock_monotonic
# ok 5 global.clock_monotonic
# # RUN global.clock_monotonic_raw ...
# # OK global.clock_monotonic_raw
# ok 6 global.clock_monotonic_raw
# # RUN global.clock_monotonic_coarse ...
# # OK global.clock_monotonic_coarse
# ok 7 global.clock_monotonic_coarse
# # PASSED: 7 / 7 tests passed.
# # Totals: pass:7 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 4 selftests: vDSO: vdso_test_clock_getres
# timeout set to 45
# selftests: vDSO: vdso_standalone_test_x86
# The time is 1718178343.398375
ok 5 selftests: vDSO: vdso_standalone_test_x86
# timeout set to 45
# selftests: vDSO: vdso_test_correctness
# TAP version 13
# 1..47
# # Warning: failed to find vsyscall getcpu
# # Testing clock_gettime for clock CLOCK_REALTIME (0)...
# # 1718178343.442886351 1718178343.442900598 1718178343.442902484
# ok 1 Test Passed.
# # Testing clock_gettime for clock CLOCK_MONOTONIC (1)...
# # 399550.364607720 399550.364610094 399550.364612050
# ok 2 Test Passed.
# # Testing clock_gettime for clock CLOCK_PROCESS_CPUTIME_ID (2)...
# # 0.003587313 0.003590008 0.003591481
# ok 3 Test Passed.
# # Testing clock_gettime for clock CLOCK_THREAD_CPUTIME_ID (3)...
# # 0.003600978 0.003602651 0.003603964
# ok 4 Test Passed.
# # Testing clock_gettime for clock CLOCK_MONOTONIC_RAW (4)...
# # 399550.322107274 399550.322109649 399550.322111534
# ok 5 Test Passed.
# # Testing clock_gettime for clock CLOCK_REALTIME_COARSE (5)...
# # 1718178343.439308591 1718178343.439308591 1718178343.439308591
# ok 6 Test Passed.
# # Testing clock_gettime for clock CLOCK_MONOTONIC_COARSE (6)...
# # 399550.361001884 399550.361001884 399550.361001884
# ok 7 Test Passed.
# # Testing clock_gettime for clock CLOCK_BOOTTIME (7)...
# # 1373293.007423993 1373293.007426368 1373293.007428254
# ok 8 Test Passed.
# # Testing clock_gettime for clock CLOCK_REALTIME_ALARM (8)...
# # 1718178343.442996909 1718178343.442999214 1718178343.443000681
# ok 9 Test Passed.
# # Testing clock_gettime for clock CLOCK_BOOTTIME_ALARM (9)...
# # 1373293.007449904 1373293.007450882 1373293.007453257
# ok 10 Test Passed.
# # Testing clock_gettime for clock CLOCK_SGI_CYCLE (10)...
# ok 11 # SKIP No such clock.
# # Testing clock_gettime for clock CLOCK_TAI (11)...
# # 1718178343.443026801 1718178343.443028198 1718178343.443030014
# ok 12 Test Passed.
# # Testing clock_gettime for clock invalid (-1)...
# ok 13 # SKIP No such clock.
# # Testing clock_gettime for clock invalid (-2147483648)...
# ok 14 # SKIP No such clock.
# # Testing clock_gettime for clock invalid (2147483647)...
# ok 15 # SKIP No such clock.
# ok 16 # SKIP No vDSO, so skipping CLOCK_REALTIME
# ok 17 # SKIP No vDSO, so skipping CLOCK_MONOTONIC
# ok 18 # SKIP No vDSO, so skipping CLOCK_PROCESS_CPUTIME_ID
# ok 19 # SKIP No vDSO, so skipping CLOCK_THREAD_CPUTIME_ID
# ok 20 # SKIP No vDSO, so skipping CLOCK_MONOTONIC_RAW
# ok 21 # SKIP No vDSO, so skipping CLOCK_REALTIME_COARSE
# ok 22 # SKIP No vDSO, so skipping CLOCK_MONOTONIC_COARSE
# ok 23 # SKIP No vDSO, so skipping CLOCK_BOOTTIME
# ok 24 # SKIP No vDSO, so skipping CLOCK_REALTIME_ALARM
# ok 25 # SKIP No vDSO, so skipping CLOCK_BOOTTIME_ALARM
# ok 26 # SKIP No vDSO, so skipping CLOCK_SGI_CYCLE
# ok 27 # SKIP No vDSO, so skipping CLOCK_TAI
# ok 28 # SKIP No vDSO, so skipping clock_gettime() tests -1
# ok 29 # SKIP No vDSO, so skipping clock_gettime() tests min
# ok 30 # SKIP No vDSO, so skipping clock_gettime() tests max
# # Testing gettimeofday...
# # 1718178343.443086 1718178343.443089 1718178343.443091
# ok 31 timezones match: minuteswest=0, dsttime=0
# # Testing getcpu...
# # CPU 3616523017:# syscall: cpu 0, node 0# vdso: cpu 0, node 0#
# ok 32 Succeeded
# # CPU 3616523017:# syscall: cpu 1, node 0# vdso: cpu 1, node 0#
# ok 33 Succeeded
# # CPU 3616523017:# syscall: cpu 2, node 0# vdso: cpu 2, node 0#
# ok 34 Succeeded
# # CPU 3616523017:# syscall: cpu 3, node 0# vdso: cpu 3, node 0#
# ok 35 Succeeded
# # CPU 3616523017:# syscall: cpu 4, node 0# vdso: cpu 4, node 0#
# ok 36 Succeeded
# # CPU 3616523017:# syscall: cpu 5, node 0# vdso: cpu 5, node 0#
# ok 37 Succeeded
# # CPU 3616523017:# syscall: cpu 6, node 0# vdso: cpu 6, node 0#
# ok 38 Succeeded
# # CPU 3616523017:# syscall: cpu 7, node 0# vdso: cpu 7, node 0#
# ok 39 Succeeded
# # CPU 3616523017:# syscall: cpu 8, node 0# vdso: cpu 8, node 0#
# ok 40 Succeeded
# # CPU 3616523017:# syscall: cpu 9, node 0# vdso: cpu 9, node 0#
# ok 41 Succeeded
# # CPU 3616523017:# syscall: cpu 10, node 0# vdso: cpu 10, node 0#
# ok 42 Succeeded
# # CPU 3616523017:# syscall: cpu 11, node 0# vdso: cpu 11, node 0#
# ok 43 Succeeded
# # CPU 3616523017:# syscall: cpu 12, node 0# vdso: cpu 12, node 0#
# ok 44 Succeeded
# # CPU 3616523017:# syscall: cpu 13, node 0# vdso: cpu 13, node 0#
# ok 45 Succeeded
# # CPU 3616523017:# syscall: cpu 14, node 0# vdso: cpu 14, node 0#
# ok 46 Succeeded
# # CPU 3616523017:# syscall: cpu 15, node 0# vdso: cpu 15, node 0#
# ok 47 Succeeded
# # Totals: pass:28 fail:0 xfail:0 xpass:0 skip:19 error:0
ok 6 selftests: vDSO: vdso_test_correctness
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/vDSO'
TAP version 13
1..6
# timeout set to 45
# selftests: vDSO: vdso_test_gettimeofday
# The time is 1718178357.227235
ok 1 selftests: vDSO: vdso_test_gettimeofday
# timeout set to 45
# selftests: vDSO: vdso_test_getcpu
# Running on CPU 10 node 0
ok 2 selftests: vDSO: vdso_test_getcpu
# timeout set to 45
# selftests: vDSO: vdso_test_abi
# TAP version 13
# 1..16
# # [vDSO kselftest] VDSO_VERSION: LINUX_2.6
# # The time is 1718178357.315664
# ok 1 __vdso_gettimeofday
# # clock_id: CLOCK_REALTIME
# # The time is 1718178357.315676746
# ok 2 __vdso_clock_gettime CLOCK_REALTIME
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 3 __vdso_clock_getres CLOCK_REALTIME
# # clock_id: CLOCK_BOOTTIME
# # The time is 1373306.880135677
# ok 4 __vdso_clock_gettime CLOCK_BOOTTIME
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 5 __vdso_clock_getres CLOCK_BOOTTIME
# # clock_id: CLOCK_TAI
# # The time is 1718178357.315711806
# ok 6 __vdso_clock_gettime CLOCK_TAI
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 7 __vdso_clock_getres CLOCK_TAI
# # clock_id: CLOCK_REALTIME_COARSE
# # The time is 1718178357.311376408
# ok 8 __vdso_clock_gettime CLOCK_REALTIME_COARSE
# # The vdso resolution is 0 4000000
# # The syscall resolution is 0 4000000
# ok 9 __vdso_clock_getres CLOCK_REALTIME_COARSE
# # clock_id: CLOCK_MONOTONIC
# # The time is 399564.237434223
# ok 10 __vdso_clock_gettime CLOCK_MONOTONIC
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 11 __vdso_clock_getres CLOCK_MONOTONIC
# # clock_id: CLOCK_MONOTONIC_RAW
# # The time is 399564.194910799
# ok 12 __vdso_clock_gettime CLOCK_MONOTONIC_RAW
# # The vdso resolution is 0 1
# # The syscall resolution is 0 1
# ok 13 __vdso_clock_getres CLOCK_MONOTONIC_RAW
# # clock_id: CLOCK_MONOTONIC_COARSE
# # The time is 399564.233069701
# ok 14 __vdso_clock_gettime CLOCK_MONOTONIC_COARSE
# # The vdso resolution is 0 4000000
# # The syscall resolution is 0 4000000
# ok 15 __vdso_clock_getres CLOCK_MONOTONIC_COARSE
# # The time in hours since January 1, 1970 is 477271
# ok 16 __vdso_time
# # Totals: pass:16 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 3 selftests: vDSO: vdso_test_abi
# timeout set to 45
# selftests: vDSO: vdso_test_clock_getres
# clock_id: CLOCK_REALTIME [PASS]
# clock_id: CLOCK_BOOTTIME [PASS]
# clock_id: CLOCK_TAI [PASS]
# clock_id: CLOCK_REALTIME_COARSE [PASS]
# clock_id: CLOCK_MONOTONIC [PASS]
# clock_id: CLOCK_MONOTONIC_RAW [PASS]
# clock_id: CLOCK_MONOTONIC_COARSE [PASS]
ok 4 selftests: vDSO: vdso_test_clock_getres
# timeout set to 45
# selftests: vDSO: vdso_standalone_test_x86
# The time is 1718178357.409346
ok 5 selftests: vDSO: vdso_standalone_test_x86
# timeout set to 45
# selftests: vDSO: vdso_test_correctness
# Warning: failed to find vsyscall getcpu
# [RUN] Testing clock_gettime for clock CLOCK_REALTIME (0)...
# 1718178357.456989189 1718178357.457004554 1718178357.457006929
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_MONOTONIC (1)...
# 399564.378711117 399564.378713491 399564.378715866
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_PROCESS_CPUTIME_ID (2)...
# 0.003684381 0.003687106 0.003688639
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_THREAD_CPUTIME_ID (3)...
# 0.003697455 0.003699018 0.003700321
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_MONOTONIC_RAW (4)...
# 399564.336209554 399564.336211928 399564.336213814
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_REALTIME_COARSE (5)...
# 1718178357.451377093 1718178357.451377093 1718178357.451377093
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_MONOTONIC_COARSE (6)...
# 399564.373070386 399564.373070386 399564.373070386
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_BOOTTIME (7)...
# 1373307.021523898 1373307.021526273 1373307.021528159
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_REALTIME_ALARM (8)...
# 1718178357.457097024 1718178357.457098002 1718178357.457099957
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_BOOTTIME_ALARM (9)...
# 1373307.021548692 1373307.021551067 1373307.021552952
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock CLOCK_SGI_CYCLE (10)...
# [OK] No such clock.
# [RUN] Testing clock_gettime for clock CLOCK_TAI (11)...
# 1718178357.457127056 1718178357.457129500 1718178357.457130478
# [OK] Test Passed.
# [RUN] Testing clock_gettime for clock invalid (-1)...
# [OK] No such clock.
# [RUN] Testing clock_gettime for clock invalid (-2147483648)...
# [OK] No such clock.
# [RUN] Testing clock_gettime for clock invalid (2147483647)...
# [OK] No such clock.
# [SKIP] No vDSO, so skipping clock_gettime64() tests
# [RUN] Testing gettimeofday...
# 1718178357.457155 1718178357.457156 1718178357.457158
# [OK] timezones match: minuteswest=0, dsttime=0
# [RUN] Testing getcpu...
# [OK] CPU 0: syscall: cpu 0, node 0 vdso: cpu 0, node 0
# [OK] CPU 1: syscall: cpu 1, node 0 vdso: cpu 1, node 0
# [OK] CPU 2: syscall: cpu 2, node 0 vdso: cpu 2, node 0
# [OK] CPU 3: syscall: cpu 3, node 0 vdso: cpu 3, node 0
# [OK] CPU 4: syscall: cpu 4, node 0 vdso: cpu 4, node 0
# [OK] CPU 5: syscall: cpu 5, node 0 vdso: cpu 5, node 0
# [OK] CPU 6: syscall: cpu 6, node 0 vdso: cpu 6, node 0
# [OK] CPU 7: syscall: cpu 7, node 0 vdso: cpu 7, node 0
# [OK] CPU 8: syscall: cpu 8, node 0 vdso: cpu 8, node 0
# [OK] CPU 9: syscall: cpu 9, node 0 vdso: cpu 9, node 0
# [OK] CPU 10: syscall: cpu 10, node 0 vdso: cpu 10, node 0
# [OK] CPU 11: syscall: cpu 11, node 0 vdso: cpu 11, node 0
# [OK] CPU 12: syscall: cpu 12, node 0 vdso: cpu 12, node 0
# [OK] CPU 13: syscall: cpu 13, node 0 vdso: cpu 13, node 0
# [OK] CPU 14: syscall: cpu 14, node 0 vdso: cpu 14, node 0
# [OK] CPU 15: syscall: cpu 15, node 0 vdso: cpu 15, node 0
ok 6 selftests: vDSO: vdso_test_correctness
make[1]: Leaving directory '/home/usama/repos/kernel/linux_mainline/tools/testing/selftests/vDSO'
