Hi Qu, On Thu, 15 Dec 2022 19:41:13 +0800, Qu Wenruo wrote: > Although the current result files "check.log" and "check.time" is enough > for human to read, it's not that easy to parse. Have you looked at the existing junit XML based report types, available via "check -R xunit ..."? junit is standardized, parsable and supported by tools such as: - https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html - https://github.com/weiwei/junitparser - https://ddiss.github.io/online-junit-parser/ - https://plugins.jenkins.io/junit/ > Thus this patch will introduce a json output to "$RESULT_BASE/check.json". > > The example output would look like this: > > { > "section": "(none)", > "fstype": "btrfs", > "start_time": 1671103264, > "arch": "x86_64", > "kernel": "6.1.0-rc8-custom+", > "results": [ > { > "testcase": "btrfs/001", > "status": "pass", > "start_time": 1671103264, > "end_time": 1671103266 > }, > { > "testcase": "btrfs/006", > "status": "pass", > "start_time": 1671103266, > "end_time": 1671103268 > }, > { > "testcase": "btrfs/007", > "status": "pass", > "start_time": 1671103268, > "end_time": 1671103271 > } > ] > } > > Which should make later parsing much easier. > > Signed-off-by: Qu Wenruo <wqu@xxxxxxxx> > --- > Reason for RFC: > > - Not crash safe > If one test case caused a crash, the "check.json" file will be an > invalid one, missing the closing "] }" string. > > - Is json really a good choice? > It may be much easier to convert to a web page, but we will still > need to parse and handle the result using another languages anyway, > like to determine a regression. I'm not opposed to adding an extra json report type, but I really think it should be plumbed into the existing common/report API. > Another alternative is .csv, and it can be much easier to handle. > (pure "echo >> $output", no need to handle the comma rule). > But for .csv, we may waste a lot of columes for things like "arch", > "kernel", "section". My preference for any new output formats, especially if they're intended for parsing, is that they're based on an existing standard/tool. E.g. https://testanything.org . Cheers, David
