TAP, the the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
$ prove ./t0005-signals.sh
./t0005-signals.sh .. ok
All tests successful.
Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)
Result: PASS
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx>
---
On Fri, May 7, 2010 at 20:25, Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> wrote:
> I'll submit an improved patch that doesn't have a confusing commit message
Here's that patch. It now also has examples of the test output
before/after and a demonstration of prove(1) running on a test.
t/README | 49 +++++++++++++++++++++++++++++++++----------------
t/test-lib.sh | 30 ++++++++++++++++++------------
2 files changed, 51 insertions(+), 28 deletions(-)
diff --git a/t/README b/t/README
index dcd3ebb..fc4bb04 100644
--- a/t/README
+++ b/t/README
@@ -18,25 +18,42 @@ The easiest way to run tests is to say "make". This runs all
the tests.
*** t0000-basic.sh ***
- * ok 1: .git/objects should be empty after git-init in an empty repo.
- * ok 2: .git/objects should have 256 subdirectories.
- * ok 3: git-update-index without --add should fail adding.
+ ok 1 - .git/objects should be empty after git init in an empty repo.
+ ok 2 - .git/objects should have 3 subdirectories.
+ ok 3 - success is reported like this
...
- * ok 23: no diff after checkout and git-update-index --refresh.
- * passed all 23 test(s)
- *** t0100-environment-names.sh ***
- * ok 1: using old names should issue warnings.
- * ok 2: using old names but having new names should not issue warnings.
- ...
-
-Or you can run each test individually from command line, like
-this:
-
- $ sh ./t3001-ls-files-killed.sh
- * ok 1: git-update-index --add to add various paths.
- * ok 2: git-ls-files -k to show killed files.
- * ok 3: validate git-ls-files -k output.
- * passed all 3 test(s)
+ ok 43 - very long name in the index handled sanely
+ # fixed 1 known breakage(s)
+ # still have 1 known breakage(s)
+ # passed all remaining 42 test(s)
+ 1..43
+ *** t0001-init.sh ***
+ ok 1 - plain
+ ok 2 - plain with GIT_WORK_TREE
+ ok 3 - plain bare
+
+Since the tests all output TAP (see http://testanything.org) they can
+be run with any TAP harness. Here's an example of paralell testing
+powered by a recent version of prove(1):
+
+ $ prove --timer --jobs 15 ./t[0-9]*.sh
+ [19:17:33] ./t0005-signals.sh ................................... ok 36 ms
+ [19:17:33] ./t0022-crlf-rename.sh ............................... ok 69 ms
+ [19:17:33] ./t0024-crlf-archive.sh .............................. ok 154 ms
+ [19:17:33] ./t0004-unwritable.sh ................................ ok 289 ms
+ [19:17:33] ./t0002-gitfile.sh ................................... ok 480 ms
+ ===( 102;0 25/? 6/? 5/? 16/? 1/? 4/? 2/? 1/? 3/? 1... )===
+
+You can also run each test individually from command line, like this:
+
+ $ sh ./t3010-ls-files-killed-modified.sh
+ ok 1 - git update-index --add to add various paths.
+ ok 2 - git ls-files -k to show killed files.
+ ok 3 - validate git ls-files -k output.
+ ok 4 - git ls-files -m to show modified files.
+ ok 5 - validate git ls-files -m output.
+ # passed all 5 test(s)
+ 1..5
You can pass --verbose (or -v), --debug (or -d), and --immediate
(or -i) command line argument to the test, or by setting GIT_TEST_OPTS
diff --git a/t/test-lib.sh b/t/test-lib.sh
index acce3d0..282314c 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -161,7 +161,7 @@ if test -n "$color"; then
*) test -n "$quiet" && return;;
esac
shift
- printf "* %s" "$*"
+ printf "%s" "$*"
tput sgr0
echo
)
@@ -170,7 +170,7 @@ else
say_color() {
test -z "$1" && test -n "$quiet" && return
shift
- echo "* $*"
+ echo "$*"
}
fi
@@ -340,25 +340,25 @@ test_have_prereq () {
test_ok_ () {
test_success=$(($test_success + 1))
- say_color "" " ok $test_count: $@"
+ say_color "" "ok $test_count - $@"
}
test_failure_ () {
test_failure=$(($test_failure + 1))
- say_color error "FAIL $test_count: $1"
+ say_color error "not ok - $test_count $1"
shift
- echo "$@" | sed -e 's/^/ /'
+ echo "$@" | sed -e 's/^/# /'
test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }
}
test_known_broken_ok_ () {
test_fixed=$(($test_fixed+1))
- say_color "" " FIXED $test_count: $@"
+ say_color "" "ok $test_count - $@ # TODO known breakage"
}
test_known_broken_failure_ () {
test_broken=$(($test_broken+1))
- say_color skip " still broken $test_count: $@"
+ say_color skip "not ok $test_count - $@ # TODO known breakage"
}
test_debug () {
@@ -390,7 +390,7 @@ test_skip () {
case "$to_skip" in
t)
say_color skip >&3 "skipping test: $@"
- say_color skip "skip $test_count: $1"
+ say_color skip "ok $test_count: # skip $1"
: true
;;
*)
@@ -601,18 +601,22 @@ test_done () {
if test "$test_fixed" != 0
then
- say_color pass "fixed $test_fixed known breakage(s)"
+ say_color pass "# fixed $test_fixed known breakage(s)"
fi
if test "$test_broken" != 0
then
- say_color error "still have $test_broken known breakage(s)"
+ say_color error "# still have $test_broken known breakage(s)"
msg="remaining $(($test_count-$test_broken)) test(s)"
else
msg="$test_count test(s)"
fi
case "$test_failure" in
0)
- say_color pass "passed all $msg"
+ # Maybe print SKIP message
+ [ -z "$skip_all" ] || skip_all=" # SKIP $skip_all"
+
+ say_color pass "# passed all $msg"
+ say "1..$test_count$skip_all"
test -d "$remove_trash" &&
cd "$(dirname "$remove_trash")" &&
@@ -621,7 +625,9 @@ test_done () {
exit 0 ;;
*)
- say_color error "failed $test_failure among $msg"
+ say_color error "# failed $test_failure among $msg"
+ say "1..$test_count"
+
exit 1 ;;
esac
--
1.7.1.dirty
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html