On Mon, Jun 14, 2010 at 21:49, Jakub Narebski <jnareb@xxxxxxxxx> wrote: > Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> writes: > >> All this series does is slightly adjust the raw text output of our >> tests so that it conforms. to the TAP standard, i.e. instead of this: >> >> $ ./t0005-signals.sh >> * ok 1: sigchain works >> * passed all 1 test(s) >> >> We get this: >> >> $ ./t0005-signals.sh >> ok 1 - sigchain works >> # passed all 1 test(s) >> 1..1 >> >> Changing the output format like this gives us the ability to run the >> Git tests with any TAP tool (like prove(1)) at no extra cost. Every >> other existing way of running the tests continues to work, it's just >> easier for machines to read the output now. I'll re-submit a version of the patch with a better commit message which addresses all of the below. > This doesn't tell us if the result of running test suite with > '--verbose' and/or with '--debug' changes, and if changes how? Is it > compatibile with TAP format so that TAP parsers understand it? It just changes in that the lines that previously said e.g. "* ok 1: sigchain works" now say "ok 1 - sigchain works". TAP parsers still understand it, since ignoring unknown garbage is part of the TAP standard. > It doesn't tell us if (as I assume) after change git test suite still > generates summary of tests in t/test-results/. Do those results > change, and if change then how? The semantics of t/test-results don't change at all since that's aggregated internally by test-lib.sh guts that aren't changed in this series. > It doesn't tell us if (as I assume) we still have the same color > output as we had before, and what we should do to have the same color > output e.g. from TAP tests in Perl. The color output is still there if you run the tests under `make test' or manually. Under TAP consumers like prove(1) the tests won't output color since the TAP Harness will set a dumb terminal. That's a good thing, since we shouldn't emit terminal escape codes with TAP. The output could be used for e.g. a HTML page instead of a terminal. > +1 > > In my opinion it is a good change, because there are many tools that > understand TAP output format (and being able to output tests in Perl > via standard Test::Simple / Test::More is also nice). Those include > e.g. Smolder and prove(1). > > It would be nice to have something like CPAN Testers > (http://cpantesters.org) but for git test suite. Junio would be able > to submit new version of git and check if it breaks on some uncommon > architecture... There's been a lot of good changes to the CPAN Testers infrastructure lately. We could probably just set up a copy of their software. Then anyone could run a smoker on some odd OS/architechture and send in test results. > P.S. I wonder who was original designer of git tests output format, > ans why it was chosen this way (and e.g. why not TAP)... It was originally discussed here: http://thread.gmane.org/gmane.comp.version-control.git/2901/focus=3076 It seems that the people involved weren't aware of TAP at the time, at least it wasn't brought up. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html