On 2020-06-16 18:52, Kees Cook wrote: > On Mon, Jun 15, 2020 at 07:07:34PM +0000, Bird, Tim wrote: >> From: Kees Cook <keescook@xxxxxxxxxxxx> >>> Note: making the plan line required differs from TAP13 and TAP14. I >>> think it's the right choice, but we should be clear. >> >> [...] >> With regards to making it optional or not, I don't have a strong >> preference. The extra info seems helpful in some circumstances. >> I don't know if it's too onerous to make it a requirement or not. >> I'd prefer if it was always there (either at the beginning or the end), >> but if there is some situation where it's quite difficult to calculate, >> then it would be best not to mandate it. I can't think of any impossible >> situations at the moment. > > I think we should require one of: > > - starting plan line > - ending plan line > - ending with something that indicates "I'm done, but I have no idea how > many tests actually ran" (Maybe "1..?") I understand the desire to be able to say "I don't know how many tests actually ran", but the use of it should be discouraged, even if available. -Frank > > To me, the point of the plan line is to be able to say "this test did, > in fact, finish". So even if some test can't even count how many tests > it _ran_, it can at least say "I am now finished". > >>> TAP13/14 makes description optional, are we making it required (I think >>> we should). There seems to be a TAP13/14 "convention" of starting >>> <description> with "- ", which I'm on the fence about it. It does make >>> parsing maybe a little easier. >> >> I would like the description to be required. >> I don't have a strong opinion on the dash. I'm OK with either one (dash >> or no dash), but we should make kselftest and KUnit consistent. > > I find the dash to be distracting -- it doesn't help me scan it, and it > doesn't help a parser (which only needs to find "#"). > >>>> Differences between kernel test result format and TAP13: >>>> - in KTAP the "# SKIP" directive is placed after the description on >>>> the test result line > > I sent a bunch of clean-ups for kselftest.h recently[1], but it looks > like we'll need more for adding "description" to skip (right now it only > prints the SKIP reason). > > [1] https://lore.kernel.org/lkml/20200611224028.3275174-1-keescook@xxxxxxxxxxxx/ > >>> Yes Documentation/*.rst Not sure on name yet, but where do kselftest >>> docs live? :) >> Documentation/dev-tools/kselftest.rst >> >> I'll put this at: Documentation/dev-tools/test-results-format.rst > > Sounds good! >
