On 2022/12/20 01:57, Darrick J. Wong wrote:
On Fri, Dec 16, 2022 at 03:05:43PM +0800, Qu Wenruo wrote:
Although "testcase" tags contain the "timestamp" element, for day-0
testing it can be hard to relate the timestamp to the tested kernel
version.
Thus this patch will add a "kernel" element to the "testcase" tag, to
indicate the kernel version we're running.
Paired with CONFIG_LOCALVERSION_AUTO=y config, it will easily show the
kernel commit we're testing.
Since we're here, also add a "arch" element, as there are more and more
aarch64 boards (From RK3399 to Apple M1) able to finish fstests in an
acceptable duration, we can no longer assume x86_64 as our only
platform.
Signed-off-by: Qu Wenruo <wqu@xxxxxxxx>
---
common/report | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/common/report b/common/report
index 4a747f8d..92586527 100644
--- a/common/report
+++ b/common/report
@@ -49,7 +49,7 @@ _xunit_make_section_report()
date_time=$(date +"%F %T")
fi
local stats="failures=\"$bad_count\" skipped=\"$notrun_count\" tests=\"$tests_count\" time=\"$sect_time\""
- local hw_info="hostname=\"$HOST\" timestamp=\"${date_time/ /T}\" "
+ local hw_info="hostname=\"$HOST\" timestamp=\"${date_time/ /T}\" arch=\"$(uname -m)\" kernel=\"$(uname -r)\""
echo "<testsuite name=\"xfstests\" $stats $hw_info >" >> $REPORT_DIR/result.xml
The original commit that added this report format was f9fde7db2f
("report: Add xunit format report generator"). Dmitry Monakhov's commit
message points out that the xml being emitted was "xunit/junit":
Footnotes:
[1] https://xunit.github.io/docs/format-xml-v2.html
[2] http://help.catchsoftware.com/display/ET/JUnit+Format
The first link is now dead, but the second link contains enough
information to find the current junit xml format:
[1] https://raw.githubusercontent.com/windyroad/JUnit-Schema/master/JUnit.xsd
Note that the xunit project appears to have diverged their report
format:
[2] https://xunit.net/docs/format-xml-v2
Great we have something solid to follow.
(Or perhaps there were multiple things called xunit?)
Either way, it's pretty obvious from common/report code that the "xunit"
code is still emitting junit xml files. I think it's important that
fstests should continue to follow that schema, so that these files can
be fed into test dashboards (yes I have one) that consume this file
format.
Totally agreed.
Regrettably, the schema does not provide for @arch or @kernel attributes
hanging off the <testsuite> element, so it's not a good idea to add
things that a strict parser could reject.
I'm totally fine with the properties method, but still curious why some
parsers would even reject elements with unknown attributes?
Isn't the idea of XML (or JSON) to be flex for newer attributes?
That said, I think it's important to record the architecture and kernel.
Probably even more attributes than that. The junit xml schema provides
for arbitrary string properties to be attached to reports; would you
mind putting these there instead?
(I want to add a few more properties now that people have started
talking about reporting again... ;))
# Properties
echo -e "\t<properties>" >> $REPORT_DIR/result.xml
echo -e "\t\t<property name=\"arch\" value=\"$(uname -m)\"/>" >> $REPORT_DIR/result.xml
echo -e "\t\t<property name=\"kernel\" value=\"$(uname -r)\"/>" >> $REPORT_DIR/result.xml
for p in "${REPORT_ENV_LIST[@]}"; do
_xunit_add_property "$p"
done
echo -e "\t</properties>" >> $REPORT_DIR/result.xml
Sounds pretty good, thanks for all the awesome advices!
Qu
--D
# Properties
--
2.38.0
