The ssh signing feature was breaking tests when the broken openssh-8.7 was used. We have now fixed that by checking for this exact case in the GPGSSH prereq and I will improve that check further in a future patch. However we are now in a situation where a broken openssh in the future will result in successfull tests but not a working git build afterwards (either not compiling in the expected feature or like in the ssh case runtime failures) resulting in a false sense of security in the tests. This patches try to improve this situation by showing which prereqs failed in the test summary and by adding an environment variable to enforce certain prereqs to succeed or abort the test otherwise. See also: https://public-inbox.org/git/xmqqv916wh7t.fsf@gitster.g/ changes sinve v2: - use a space separated list for GIT_TES_REQUIRED_PREREQ like we do for GIT_SKIP_TESTS - use BAIL_OUT() insted of just error() - make BAIL_OUT() print errors even when used within prereq context changes since v1: - use \012 instead of \n for possible portability reasons - fix typo in commit msg Fabian Stelzer (3): test-lib: show missing prereq summary test-lib: introduce required prereq for test runs test-lib: make BAIL_OUT() work in tests and prereq t/README | 6 ++++++ t/aggregate-results.sh | 17 +++++++++++++++++ t/test-lib-functions.sh | 11 +++++++++++ t/test-lib.sh | 21 +++++++++++++++++---- 4 files changed, 51 insertions(+), 4 deletions(-) Range-diff against v2: 1: 69e77cd854 ! 1: 35c92671e5 test-lib: show missing prereq summary @@ t/aggregate-results.sh: do + tr -s "," "\n" | + grep -v '^$' | + sort -u | -+ paste -s -d ',') -+ if test -n $unique_missing_prereq ++ paste -s -d ' ') ++ if test -n "$unique_missing_prereq" + then + printf "\nmissing prereq: $unique_missing_prereq\n\n" + fi 2: 12bd18c5ce ! 2: d6a53f0980 test-lib: introduce required prereq for test runs @@ Commit message test-lib: introduce required prereq for test runs In certain environments or for specific test scenarios we might expect a - specific prerequisite check to be succeed. Therefore we would like to - trigger an error when running our tests if this is not the case. + specific prerequisite check to succeed. Therefore we would like to abort + running our tests if this is not the case. To remedy this we add the environment variable GIT_TEST_REQUIRE_PREREQ - which can be set to a comma separated list of prereqs. If one of these + which can be set to a space separated list of prereqs. If one of these prereq tests fail then the whole test run will abort. Signed-off-by: Fabian Stelzer <fs@xxxxxxxxxxxx> @@ t/README: explicitly providing repositories when accessing submodule objects is complete or needs to be abandoned for whatever reason (in which case the migrated codepaths still retain their performance benefits). -+GIT_TEST_REQUIRE_PREREQ=<list> allows specifying a comma speparated list of ++GIT_TEST_REQUIRE_PREREQ=<list> allows specifying a space speparated list of +prereqs that are required to succeed. If a prereq in this list is triggered by +a test and then fails then the whole test run will abort. This can help to make +sure the expected tests are executed and not silently skipped when their @@ t/test-lib-functions.sh: test_have_prereq () { prerequisite=${negative_prereq:+!}$prerequisite + + # Abort if this prereq was marked as required -+ if test -n $GIT_TEST_REQUIRE_PREREQ ++ if test -n "$GIT_TEST_REQUIRE_PREREQ" + then -+ case ",$GIT_TEST_REQUIRE_PREREQ," in -+ *,$prerequisite,*) -+ error "required prereq $prerequisite failed" ++ case " $GIT_TEST_REQUIRE_PREREQ " in ++ *" $prerequisite "*) ++ BAIL_OUT "required prereq $prerequisite failed" + ;; + esac + fi -: ---------- > 3: de21c484d6 test-lib: make BAIL_OUT() work in tests and prereq base-commit: cd3e606211bb1cf8bc57f7d76bab98cc17a150bc -- 2.31.1
