Currently, test result files are only stored in test-results/*.counts if
$HARNESS_ACTIVE is not set. This dates from 8ef1abe550 (test-lib: Don't
write test-results when HARNESS_ACTIVE, 2010-08-11), where the
assumption was that if someone were using a test harness like prove,
that would track results and the count files wouldn't be required.
However, as of 49da404070 (test-lib: show missing prereq summary,
2021-11-20), those files also store the list of git test prerequisites
that were missing during the test run, which isn't something that a
generic test harness like prove can provide.
To allow folk using test harnesses to access the lists of missing
prerequisites, add a --counts argument to test-lib that will keep these
counts files even if a test harness is in use. This means that a
subsequent call of, say, `make -C t aggregate-results` will report
useful information.
It might be preferable to do make a wider-ranging change, including
storing the missing prerequisites separately from the count files, so
the results can be reported regardless of whether the success/failure
counts are wanted, but that would be more disruptive and more work for
relatively little gain.
Signed-off-by: Adam Dinwoodie <adam@xxxxxxxxxxxxx>
---
I submitted this as an RFC back in December, and received no comments,
so I'm submitting this as an actual patch now. My key concern was the
final paragraph above -- embedding using the "count" files for something
other than counts -- but I've mostly convinced myself that refactoring
this code to separate that out is unlikely to actually cause significant
pain.
t/test-lib.sh | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 6db377f68b..bbd9ee0e34 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -157,6 +157,8 @@ parse_option () {
local opt="$1"
case "$opt" in
+ -c|--c|--co|--cou|--coun|--count|--counts)
+ record_counts=t ;;
-d|--d|--de|--deb|--debu|--debug)
debug=t ;;
-i|--i|--im|--imm|--imme|--immed|--immedi|--immedia|--immediat|--immediate)
@@ -1282,7 +1284,7 @@ test_done () {
finalize_test_output
- if test -z "$HARNESS_ACTIVE"
+ if test -z "$HARNESS_ACTIVE" || test -n "$record_counts"
then
mkdir -p "$TEST_RESULTS_DIR"
