On 2023/2/15 02:46, Darrick J. Wong wrote:
On Tue, Dec 20, 2022 at 09:14:07AM +0800, Qu Wenruo wrote:
On 2022/12/20 08:01, Darrick J. Wong wrote:
From: Darrick J. Wong <djwong@xxxxxxxxxx>
Generate the section report between tests so that the summary report
always reflects the outcome of the most recent test. Two usecases are
envisioned here -- if a cluster-based test runner anticipates that the
testrun could crash the VM, they can set REPORT_DIR to (say) an NFS
mount to preserve the intermediate results. If the VM does indeed
crash, the scheduler can examine the state of the crashed VM and move
the tests to another VM. The second usecase is a reporting agent that
runs in the VM to upload live results to a test dashboard.
Signed-off-by: Darrick J. Wong <djwong@xxxxxxxxxx>
---
check | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/check b/check
index 1ff0f44af1..70a0b537b1 100755
--- a/check
+++ b/check
@@ -842,6 +842,16 @@ function run_section()
fi
seqres="$REPORT_DIR/$seqnum"
+ # Generate the entire section report with whatever test results
+ # we have so far. Leave the $sect_time parameter empty so that
+ # it's a little more obvious that this test run is incomplete.
+ if $do_report; then
+ local sect_now=`_wallclock`
+ _make_section_report "$section" "${#try[*]}" \
+ "${#bad[*]}" "${#notrun[*]}" \
+ "" &> /dev/null
+ fi
+
In fact that's exactly what I just wanted to do next...
However one problem I can't work around is the timestamp.
Currently our timestamp is the finish time, with this change our timestamp
is the start time.
Er... what do you mean exactly by start and finish time? The most
recent test? The entire run?
Personally speaking, I'd prefer start/finish time of the whole
suite/section at least.
From check:
if [ -w /dev/kmsg ]; then
export date_time=`date +"%F %T"`
echo "run fstests $seqnum at $date_time" > /dev/kmsg
...
fi
From common/report:
if [ -z "$date_time" ]; then
date_time=$(date +"%F %T")
fi
...timestamp="${date_time/ /T}">
The "date_time" variable can be set by ./check just prior to starting
each test, but only if /dev/kmsg is a writable file. If it's set, then
the timestamp in the xml file reflects the start time of the most recent
test.
If date_time is not set, then the timestamp in the xml file records the
point in time when the report is generated... which I suppose is a good
enough estimate if we failed to record date_time when the test was
started. >
I'm not sure if there is any strong definition on the timestamp, but
personally speaking, timestamp for the start time makes more sense to me.
But there may be some users relying on the timestamp to be the ending time?
The junit xml schema says that the timestamp should be "when the test
was executed". I think ./check ought to be setting date_time regardless
of /dev/kmsg. ATM my test dashboard computes the difference between now
and the timestamp= attribute of the last report received to decide if it
should flag a test VM as possibly stalled.
Though to undercut everything I just wrote, the junit xml schema defines
testsuite and testcase as distinct entities and then refers to "the
test".
Given that fstests doesn't strictly follow that schema anyway, I guess
we can define the timestamp as we want, or add more timestamp
attributes?
<testsuite suite_timestamp="..." timestamp="..." report_timestamp="..."...>
OK, then this looks much better.
Thanks,
Qu
--D
Thanks,
Qu
mkdir -p $RESULT_DIR
rm -f ${RESULT_DIR}/require_scratch*
rm -f ${RESULT_DIR}/require_test*
