On 8/1/24 10:27 PM, Shuah Khan wrote: > On 8/1/24 10:27, Shuah Khan wrote: >> On 8/1/24 02:42, Muhammad Usama Anjum wrote: >>> On 7/31/24 9:57 PM, Shuah Khan wrote: >>>> On 7/31/24 07:39, Muhammad Usama Anjum wrote: >>>>> Don't print that 88 sub-tests are going to be executed, but then skip. >>>>> This is against TAP compliance. Instead check pre-requisites first >>>>> before printing total number of tests. >>>> >>>> Does TAP clearly mention this? >>> Yes from https://testanything.org/tap-version-13-specification.html >>> >>> Skipping everything >>> This listing shows that the entire listing is a skip. No tests were run. >>> >>> TAP version 13 >>> 1..0 # skip because English-to-French translator isn't installed >> > > One more thing on TAP compliance - we don't strive to be TAP compliant > as it doesn't meet our special needs. > > It is important to keep the how many tests could be run to improve testing > coverage. > > Refer to: https://www.kernel.org/doc/html/latest/dev-tools/ktap.html > > "The Linux Kernel largely uses TAP output for test results. However, Kernel > testing frameworks have special needs for test results which don’t align with > the original TAP specification. Thus, a “Kernel TAP” (KTAP) format is > specified > to extend and alter TAP to support these use-cases. This specification > describes > the generally accepted format of KTAP as it is currently used in the kernel." > > I appreciate the effort you are putting into improving the reports. I request > that you refer to the above document and also keep in mind what would help us > improve our testing over focusing just on reports and compliance. That makes sense. I'll work on improving the testing going forward. > > thanks, > -- Shuah > -- BR, Muhammad Usama Anjum
