Such situation may simply show that what was tested is now fixed and that it's juste the test annotation which need to be adapted, but can be a sign that something else is broken. Reporting the exact result (failure/success, known-to-fail/expect-to-succeed) make the testsuite more useful and allow to use more efficiently git-bisect or other automated testing tools. Signed-off-by: Luc Van Oostenryck <luc.vanoostenryck@xxxxxxxxx> --- validation/test-suite | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/validation/test-suite b/validation/test-suite index df5a7c60..03421a24 100755 --- a/validation/test-suite +++ b/validation/test-suite @@ -161,15 +161,26 @@ do_test() test_failed=1 fi + get_tag "check-known-to-fail" $file + if [ "$?" -eq "0" ]; then + must_fail=1 + known_ko_tests=`expr $known_ko_tests + 1` + else + must_fail=0 + fi + if [ "$test_failed" -eq "1" ]; then ko_tests=`expr $ko_tests + 1` - get_tag "check-known-to-fail" $file - if [ "$?" -eq "0" ]; then + if [ "$must_fail" -eq "1" ]; then echo "info: test '$file' is known to fail" - known_ko_tests=`expr $known_ko_tests + 1` fi return 1 else + if [ "$must_fail" -eq "1" ]; then + echo "info: test '$file' is known to fail but succeed!" + ko_tests=`expr $ko_tests + 1` + return 1 + fi ok_tests=`expr $ok_tests + 1` return 0 fi -- 2.10.1 -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
