On 05.11.2021 12:11, Junio C Hamano wrote:
Adam Dinwoodie <adam@xxxxxxxxxxxxx> writes:
This is probably a much broader conversation. I remember when I first
started packaging Git for Cygwin, I produced a release that didn't
have support for HTTPS URLs due to a missing dependency in my build
environment. The build and test suite all passed -- it assumed I just
wanted to build a release that didn't have HTTPS support -- so some
relatively critical function was silently skipped. I don't know how to
avoid that sort of issue other than relying on (a) user bug (or at
least missing function) reports and (b) folk building Git for
themselves/others periodically going through the output of the
configure scripts and the skipped subtests to make sure only expected
things get missed; neither of those options seem great to me.
I agree with you that there needs a good way to enumerate what the
unsatisfied prerequisites for a particular build are. That would
have helped in your HTTPS situation.
Sorry for not replying earlier. I've been sick the last couple of days
and only slowly getting up to speed again. I will improve the prereq
tests in a new commit in the other patch series still in progress that
i'll shortly reroll.
As for the general prereq issue i ran into that as well during
development. When you depend on other patches / a specific version of
ssh-keygen for git I always have to remember to set the path correctly
or the tests might silently be ignored by the missing prereq. Usually
not a problem for single test runs, but when i run the full suite before
sending something.
So, here's a simple rfc patch to maybe start with addressing this issue.
>From 0e7e57e546ec969d31094405aecafd1b1f3cf4d8 Mon Sep 17 00:00:00 2001
From: Fabian Stelzer <fs@xxxxxxxxxxxx>
Date: Fri, 12 Nov 2021 16:41:30 +0100
Subject: [RFC PATCH 1/2] test-lib: show failed prereq summary
Add failed prereqs to the test results.
Aggregate and then show them with the totals.
Signed-off-by: Fabian Stelzer <fs@xxxxxxxxxxxx>
---
t/aggregate-results.sh | 12 ++++++++++++
t/test-lib.sh | 4 ++++
2 files changed, 16 insertions(+)
diff --git a/t/aggregate-results.sh b/t/aggregate-results.sh
index 7913e206ed..ad531cc75d 100755
--- a/t/aggregate-results.sh
+++ b/t/aggregate-results.sh
@@ -6,6 +6,7 @@ success=0
failed=0
broken=0
total=0
+missing_prereq=
while read file
do
@@ -30,10 +31,21 @@ do
broken=$(($broken + $value)) ;;
total)
total=$(($total + $value)) ;;
+ missing_prereq)
+ missing_prereq="$missing_prereq $value" ;;
esac
done <"$file"
done
+if test -n "$missing_prereq"
+then
+ unique_missing_prereq=$(
+ echo $missing_prereq | tr -s "," | \
+ sed -e 's/ //g' -e 's/^,//' -e 's/,$//' -e 's/,/\n/g' \
+ | sort | uniq | paste -s -d ',')
+ printf "\nmissing prereq: $unique_missing_prereq\n\n"
+fi
+
if test -n "$failed_tests"
then
printf "\nfailed test(s):$failed_tests\n\n"
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 2679a7596a..472387afec 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -669,6 +669,8 @@ test_fixed=0
test_broken=0
test_success=0
+test_missing_prereq=
+
test_external_has_tap=0
die () {
@@ -1068,6 +1070,7 @@ test_skip () {
then
of_prereq=" of $test_prereq"
fi
+ test_missing_prereq="$missing_prereq,$test_missing_prereq"
skipped_reason="missing $missing_prereq${of_prereq}"
fi
@@ -1175,6 +1178,7 @@ test_done () {
fixed $test_fixed
broken $test_broken
failed $test_failure
+ missing_prereq $test_missing_prereq
EOF
fi
--
2.31.1
>From d13d4c8ccbd832e1d62044b18b8b771f6586ee2a Mon Sep 17 00:00:00 2001
From: Fabian Stelzer <fs@xxxxxxxxxxxx>
Date: Fri, 12 Nov 2021 16:43:18 +0100
Subject: [RFC PATCH 2/2] test-lib: introduce required prereq for test runs
Allows setting GIT_TEST_REQUIRE_PREREQ to a number of prereqs that must
succeed for this run. Otherwise the test run will abort.
Signed-off-by: Fabian Stelzer <fs@xxxxxxxxxxxx>
---
t/test-lib-functions.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index eef2262a36..d65995cd15 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -669,6 +669,14 @@ test_have_prereq () {
satisfied_this_prereq=t
;;
*)
+ if ! test -z $GIT_TEST_REQUIRE_PREREQ
+ then
+ case ",$GIT_TEST_REQUIRE_PREREQ," in
+ *,$prerequisite,*)
+ error "required prereq $prerequisite failed"
+ ;;
+ esac
+ fi
satisfied_this_prereq=
esac
--
2.31.1
