> Hi Zhangjin,
>
> On Mon, Jun 19, 2023 at 02:52:31PM +0800, Zhangjin Wu wrote:
> > Hi, Willy
> >
> > Here is the v2 of our old patchset about test report [1].
> >
> > The trailing '\r' fixup has been merged, so, here only resend the left
> > parts with an additional patch to restore the failed tests print.
> >
> > This patchset is rebased on the dev.2023.06.14a branch of linux-rcu [2].
> >
> > Tests have passed for 'x86 run':
> >
> > 138 test(s) passed, 0 skipped, 0 failed.
> > See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.out
> (...)
> > 2. selftests/nolibc: always print the path to test log file
> >
> > Always print the path to test log file, but move it to a new line to
> > avoid annoying people when the test pass without any failures.
>
> I'm still really missing the (s+f > 0) test I added which was a time saver
> for me, because I could trivially check in the output reports which ones
> were totally OK and which ones required attention. Sure I could also start
> to grep for "passed," | grep -v " 0 skipped, 0 failed" but that's quite a
> pain, really.
>
> I'm going to merge your series anyway otherwise we'll continue to bikeshed
> for many weeks and I know how annoying it is to keep unmerged series. But
> I would like that we find a solution that satisfies everyone.
>
> Maybe one possibility would be to add a "status" at the end of the line
> that emits "success", "warning", "failure" depending on the highest level
> reached like this:
>
> 138 test(s) passed, 0 skipped, 0 failed => status: success
> 136 test(s) passed, 2 skipped, 0 failed => status: warning
> 136 test(s) passed, 1 skipped, 1 failed => status: failure
>
> This way it's easy to grep -v "status: success" or grep "status: failure"
> to instantly get the corresponding details and also grep for them from
> multiple files.
Ok, it will be a further step, based on your new awk script, it is not
that hard.
This should work:
REPORT ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{f++;print} /\[SKIPPED\][\r]*$$/{s++} \
END{ printf("\n%d test(s): %d passed, %d skipped, %d failed => status: ", p+s+f, p, s, f); \
if (f) printf("failure\n"); else if (s) printf("warning\n"); else printf("success\n");; \
printf("\nSee all results in %s\n", ARGV[1]); }'
It reports something like this:
...
Total number of errors: 0
Exiting with status 0
143 test(s): 140 passed, 3 skipped, 0 failed => status: warning
See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/build/x86/run.out
Two newlines are added around the summary line, so, it is not crowded as
before. If this is ok, will send a new patch on your new branch.
Thanks very much.
Best regards,
Zhangjin
>
> Thanks!
> Willy
