On Tue, Nov 03, 2020 at 01:33:53PM +0200, Andy Shevchenko wrote: > On Tue, Nov 03, 2020 at 04:40:49PM +0530, Arpitha Raghunandan wrote: > > Convert test lib/test_printf.c to KUnit. More information about > > KUnit can be found at: > > https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html. > > KUnit provides a common framework for unit tests in the kernel. > > KUnit and kselftest are standardizing around KTAP, converting this > > test to KUnit makes this test output in KTAP which we are trying to > > make the standard test result format for the kernel. More about > > the KTAP format can be found at: > > https://lore.kernel.org/linux-kselftest/CY4PR13MB1175B804E31E502221BC8163FD830@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/. > > I ran both the original and converted tests as is to produce the > > output for success of the test in the two cases. I also ran these > > tests with a small modification to show the difference in the output > > for failure of the test in both cases. The modification I made is: > > - test("127.000.000.001|127.0.0.1", "%pi4|%pI4", &sa.sin_addr, &sa.sin_addr); > > + test("127-000.000.001|127.0.0.1", "%pi4|%pI4", &sa.sin_addr, &sa.sin_addr); > > > > Original test success: > > [ 0.540860] test_printf: loaded. > > [ 0.540863] test_printf: random seed = 0x5c46c33837bc0619 > > [ 0.541022] test_printf: all 388 tests passed > > > > Original test failure: > > [ 0.537980] test_printf: loaded. > > [ 0.537983] test_printf: random seed = 0x1bc1efd881954afb > > [ 0.538029] test_printf: vsnprintf(buf, 256, "%pi4|%pI4", ...) wrote '127.000.000.001|127.0.0.1', expected '127-000.000.001|127.0.0.1' > > [ 0.538030] test_printf: kvasprintf(..., "%pi4|%pI4", ...) returned '127.000.000.001|127.0.0.1', expected '127-000.000.001|127.0.0.1' > > [ 0.538124] test_printf: failed 2 out of 388 tests > > [ 0.538125] test_printf: random seed used was 0x1bc1efd881954afb > > > > Converted test success: > > # Subtest: printf > > 1..25 > > ok 1 - test_basic > > ok 2 - test_number > > ok 3 - test_string > > ok 4 - plain > > ok 5 - null_pointer > > ok 6 - error_pointer > > ok 7 - invalid_pointer > > ok 8 - symbol_ptr > > ok 9 - kernel_ptr > > ok 10 - struct_resource > > ok 11 - addr > > ok 12 - escaped_str > > ok 13 - hex_string > > ok 14 - mac > > ok 15 - ip > > ok 16 - uuid > > ok 17 - dentry > > ok 18 - struct_va_format > > ok 19 - time_and_date > > ok 20 - struct_clk > > ok 21 - bitmap > > ok 22 - netdev_features > > ok 23 - flags > > ok 24 - errptr > > ok 25 - fwnode_pointer > > ok 1 - printf > > > > Converted test failure: > > # Subtest: printf > > 1..25 > > ok 1 - test_basic > > ok 2 - test_number > > ok 3 - test_string > > ok 4 - plain > > ok 5 - null_pointer > > ok 6 - error_pointer > > ok 7 - invalid_pointer > > ok 8 - symbol_ptr > > ok 9 - kernel_ptr > > ok 10 - struct_resource > > ok 11 - addr > > ok 12 - escaped_str > > ok 13 - hex_string > > ok 14 - mac > > # ip: EXPECTATION FAILED at lib/printf_kunit.c:82 > > vsnprintf(buf, 256, "%pi4|%pI4", ...) wrote '127.000.000.001|127.0.0.1', expected '127-000.000.001|127.0.0.1' > > # ip: EXPECTATION FAILED at lib/printf_kunit.c:124 > > kvasprintf(..., "%pi4|%pI4", ...) returned '127.000.000.001|127.0.0.1', expected '127-000.000.001|127.0.0.1' > > not ok 15 - ip > > ok 16 - uuid > > ok 17 - dentry > > ok 18 - struct_va_format > > ok 19 - time_and_date > > ok 20 - struct_clk > > ok 21 - bitmap > > ok 22 - netdev_features > > ok 23 - flags > > ok 24 - errptr > > ok 25 - fwnode_pointer > > not ok 1 - printf > > Better, indeed. > > But can be this improved to have a cumulative statistics, like showing only > number of total, succeeded, failed with details of the latter ones? Is that the proper test output format? We have a standard... thanks, greg k-h
