On Tue, Dec 20, 2022 at 09:14:07AM +0800, Qu Wenruo wrote:
>
>
> On 2022/12/20 08:01, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@xxxxxxxxxx>
> >
> > Generate the section report between tests so that the summary report
> > always reflects the outcome of the most recent test. Two usecases are
> > envisioned here -- if a cluster-based test runner anticipates that the
> > testrun could crash the VM, they can set REPORT_DIR to (say) an NFS
> > mount to preserve the intermediate results. If the VM does indeed
> > crash, the scheduler can examine the state of the crashed VM and move
> > the tests to another VM. The second usecase is a reporting agent that
> > runs in the VM to upload live results to a test dashboard.
> >
> > Signed-off-by: Darrick J. Wong <djwong@xxxxxxxxxx>
> > ---
> > check | 10 ++++++++++
> > 1 file changed, 10 insertions(+)
> >
> >
> > diff --git a/check b/check
> > index 1ff0f44af1..70a0b537b1 100755
> > --- a/check
> > +++ b/check
> > @@ -842,6 +842,16 @@ function run_section()
> > fi
> > seqres="$REPORT_DIR/$seqnum"
> > + # Generate the entire section report with whatever test results
> > + # we have so far. Leave the $sect_time parameter empty so that
> > + # it's a little more obvious that this test run is incomplete.
> > + if $do_report; then
> > + local sect_now=`_wallclock`
> > + _make_section_report "$section" "${#try[*]}" \
> > + "${#bad[*]}" "${#notrun[*]}" \
> > + "" &> /dev/null
> > + fi
> > +
>
> In fact that's exactly what I just wanted to do next...
>
> However one problem I can't work around is the timestamp.
>
> Currently our timestamp is the finish time, with this change our timestamp
> is the start time.
Er... what do you mean exactly by start and finish time? The most
recent test? The entire run?
>From check:
if [ -w /dev/kmsg ]; then
export date_time=`date +"%F %T"`
echo "run fstests $seqnum at $date_time" > /dev/kmsg
...
fi
>From common/report:
if [ -z "$date_time" ]; then
date_time=$(date +"%F %T")
fi
...timestamp="${date_time/ /T}">
The "date_time" variable can be set by ./check just prior to starting
each test, but only if /dev/kmsg is a writable file. If it's set, then
the timestamp in the xml file reflects the start time of the most recent
test.
If date_time is not set, then the timestamp in the xml file records the
point in time when the report is generated... which I suppose is a good
enough estimate if we failed to record date_time when the test was
started.
> I'm not sure if there is any strong definition on the timestamp, but
> personally speaking, timestamp for the start time makes more sense to me.
> But there may be some users relying on the timestamp to be the ending time?
The junit xml schema says that the timestamp should be "when the test
was executed". I think ./check ought to be setting date_time regardless
of /dev/kmsg. ATM my test dashboard computes the difference between now
and the timestamp= attribute of the last report received to decide if it
should flag a test VM as possibly stalled.
Though to undercut everything I just wrote, the junit xml schema defines
testsuite and testcase as distinct entities and then refers to "the
test".
Given that fstests doesn't strictly follow that schema anyway, I guess
we can define the timestamp as we want, or add more timestamp
attributes?
<testsuite suite_timestamp="..." timestamp="..." report_timestamp="..."...>
--D
> Thanks,
> Qu
>
> > mkdir -p $RESULT_DIR
> > rm -f ${RESULT_DIR}/require_scratch*
> > rm -f ${RESULT_DIR}/require_test*
> >
