On 6/12/24 1:27 AM, Shuah Khan wrote: > On 6/10/24 02:32, Muhammad Usama Anjum wrote: >> Although "TAP" word is being used already in documentation, but it hasn't >> been defined in informative way for developers that how to write TAP >> conformant tests and what are the benefits. Write a short brief about it. >> >> Signed-off-by: Muhammad Usama Anjum <usama.anjum@xxxxxxxxxxxxx> >> --- >> Documentation/dev-tools/kselftest.rst | 8 ++++++++ >> 1 file changed, 8 insertions(+) >> >> diff --git a/Documentation/dev-tools/kselftest.rst >> b/Documentation/dev-tools/kselftest.rst >> index dcf634e411bd9..b579f491f3e97 100644 >> --- a/Documentation/dev-tools/kselftest.rst >> +++ b/Documentation/dev-tools/kselftest.rst >> @@ -228,6 +228,14 @@ In general, the rules for selftests are >> * Don't cause the top-level "make run_tests" to fail if your feature is >> unconfigured. >> + * The output of tests must conform to the TAP standard to ensure high >> + testing quality and to capture failures/errors with specific details. >> + The kselftest.h and kselftest_harness.h headers provide wrappers for >> + outputting test results such as pass, fail, or skip etc. These wrappers >> + should be used instead of reinventing the wheel or using raw printf and > > When there are multiple tests supported by a test suite, it doesn't > make sense to convert every single print statement. That is the reason > why we have the run_tests wrapper that prints the summary in TAP format. > > I would rephrase this to say "should be used for pass, fail, exit, and > skip messages". Makes sense. I'll update. > > This rule is applicable to only the tests that are in the selftests Makefile > and included in the default run. > > There is no need to convert every single print statement in tests. > >> + exit statements. CI systems can easily parse TAP output messages to >> + detect test failures. > > I would rather see people spending energy writing new tests and fixing > warnings. TAP conversions need to be focused if they add value. Some tests don't print pass/fail and just return status by exit value. Adding pass/fail statements there really add value by exposing what went wrong. So it can be reproduced in debug environment to fix it. > > thanks, > -- Shuah > -- BR, Muhammad Usama Anjum
