Heya,
On Tue, Aug 17, 2010 at 04:43, Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> wrote:
> Change the sanity tests in t/t0000-basic.sh to not to run a passing
> TODO test unless the TEST_PASSING_TODO environment variable is set.
>
> The motivation is to have nothing out of the ordinary on a normal test
> run for test smoking purposes.
>
> If every normal test run has a passing TODO you're more likely to turn
> a blind eye to it and not to investigate cases where things really are
> passing unexpectedly.
Thanks, this has bothered me ever since I first ran the test suite.
While I understand that the test suite needs testing to, it should be
done in such a way that it doesn't produce false positives. That is,
my preference would be to still keep the test in the suite as-is, but
somehow get it to show up as a passing test instead of a fixed one.
E.g., something like:
test_in_new_harness 0 1 0 0 '
test_expect_failure 'pretend we have fixed a known breakage' '
:
'
'
Where there '0 1 0 0' stands for '0 failures, 1 fixed, 0 broken, 0
success'. Not sure there's any other use for a test_in_new_harness
function though...
PS: To clarify, test_in_new_harness would not change any of the
counters (test_failure, test_count, test_fixed, test_broken,
test_success), but would instead compare those against the values
specified as argument to test_in_new_harness.
PSPS: I have no idea how easy it would be to implement something like this.
--
Cheers,
Sverre Rabbelier
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html