On 2/2/22 5:31 AM, Naresh Kamboju wrote: > Linaro started doing Linux kernel Functional Validation (LKFT). > As part of LKFT recently we have enabled CONFIG_OF_UNITTEST=y in our > daily test CI. > > The output of the test looks as below. The current problem is that we > have a hard time to see (grep) pass/fail for each individual test. We > only see a summary at the end with x pass and y fails. The FAIL messages are printed at loglevel KERN_ERR. The pass messages are printed at loglevel KERN_DEBUG. To see the pass messages, set the loglevel to allow debug output. Unfortunately this can add lots of debug output, unless you use dynamic debug to only enable debug for drivers/of/unittest.o. There are only a few other pr_debug() messages in unittest. I think a better solution would be to add a config option, something like CONFIG_OF_UNITTEST_VERBOSE, that would print the pass messages at loglevel KERN_ERR. I'll submit a patch for that and see what the review responses are. > We would like to get your opinion of how hard it would be to include > that in the output per test. Maybe like TAP version 14? > Another question would be how hard do you think it would be to rewrite > this to a kunit test, if even applicable? I have provided the kunit > output links at the end of this email. Devicetree unittests were suggested as a good candidate as a first test to convert to kunit when kunit was implemented. Brendan tried to convert it, and we quickly saw that it was not a good candidate. Devicetree unittests do not fit the unit test mold; they are a very different creature. Brendan has a good term for this type of test (Brendan, was it "acceptance" test?). > > > Test output: > ------------ > [ 0.000000] Booting Linux on physical CPU 0x0000000100 [0x410fd033] > [ 0.000000] Linux version 5.17.0-rc1-next-20220127 > (tuxmake@tuxmake) (aarch64-linux-gnu-gcc (Debian 11.2.0-9) 11.2.0, GNU > ld (GNU Binutils for Debian) 2.37) #1 SMP PREEMPT @1643255563 > [ 0.000000] Machine model: ARM Juno development board (r2) > > <trimmed output> > > [ 3.285226] ### dt-test ### start of unittest - you will see error messages > [ 3.293269] ### dt-test ### EXPECT \ : Duplicate name in > testcase-data, renamed to \"duplicate-name#1\" > [ 3.293456] Duplicate name in testcase-data, renamed to \"duplicate-name#1\" > [ 3.313367] ### dt-test ### EXPECT / : Duplicate name in > testcase-data, renamed to \"duplicate-name#1\" > [ 3.314709] ### dt-test ### EXPECT \ : OF: > /testcase-data/phandle-tests/consumer-a: could not get > #phandle-cells-missing for /testcase-data/phandle-tests/provider1 > [ 3.323968] OF: /testcase-data/phandle-tests/consumer-a: could not > get #phandle-cells-missing for /testcase-data/phandle-tests/provider1 > > <trimmed output> > > [ 5.118400] ### dt-test ### EXPECT / : OF: overlay: ERROR: multiple > fragments add and/or delete node > /testcase-data-2/substation@100/motor-1/electric > [ 5.121358] atkbd serio1: keyboard reset failed on 1c070000.kmi > [ 5.134160] ### dt-test ### end of unittest - 257 passed, 0 failed > > > Ref: > Full test output of of-unittest > https://lkft.validation.linaro.org/scheduler/job/4458582#L1019 > https://lkft.validation.linaro.org/scheduler/job/4404330#L428 > > Kunit example test output that we are running in our daily CI loop. > https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-5.10.y/build/v5.10.70/testrun/5965109/suite/kunit/tests/ > > Kunit Full test logs: > https://lkft.validation.linaro.org/scheduler/job/3643324 > > https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-5.10.y/build/v5.10.70/testrun/5965109/suite/kunit/test/kunit_log_test/log > > > -- > Linaro LKFT > https://lkft.linaro.org >
