On Tue, Mar 13, 2012 at 05:53:32PM -0400, Jeff King wrote: > The reason is that the individual tests do not verify all of the > preconditions themselves, but rather build on each other. Right. I added a note based on this sentence in the test description. > In an ideal world, each test snippet would be totally independent and > check its preconditions. That would give us an accurate count of how > many tests actually passed or failed. But fundamentally we only care > about "did they all succeed or not?", which the current script does tell > us (either test 2 fails, or if it succeeds, then we have checked the > precondition for test 4). And the tests end up way shorter, because we > don't repeat the preconditions over and over. > > If you want to try to make the tests more robust, you can (for example, > you can tighten the precondition on 4 to check "does it give the right > answer with the right protocol" instead of just "does it ever give us > the right answer"). But personally, I'm not sure it's worth that much > effort. Yeah. > > >Should they actually say "# SKIP ..." to tell prove what's going on? I > > >don't know very much about TAP. > > # SKIP is used when skipping individual tests (IIUC), but when we > > skip a group of tests, we simply jump over them and this message is > > purely informative output that is not interpreted by the harness. > > Just looking at test-lib.sh, it seems like we output "# SKIP" when we do > skip_all. But I think you would have to give a count of which tests you > skipped (e.g., try "./t5541-http-push.sh" to see its TAP output). Which > means when skipping a subset, you'd have to deal with test numbering, > which is a pain. So it's probably not worth worrying about. Skipped test numbering could done automatically by using test prereqs, but (after actually doing that and discarding) I agree that it isn't worth the trouble. Jonathan Nieder wrote: > Nice idea, but shouldn't this description be in test_description so I > can view it by running "sh t0303-credential-external.sh --help"? Done. Updated patches follow. (This time I tested with GIT_TEST_CREDENTIAL_HELPER=cache GIT_TEST_CREDENTIAL_HELPER_TIMEOUT="cache --timeout=1,3" and things seem to work as expected.) Zbyszek -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html