On 6/2/2018 5:19 PM, Jakub Narebski wrote:
Derrick Stolee <stolee@xxxxxxxxx> writes:
Do we have a way to run individual steps of the test suite? I am
unfamiliar with that process.
The t/README describes three such ways in "Skipping Tests" section:
- GIT_SKIP_TESTS environment variable, which can either can match the
"t[0-9]{4}" part to skip the whole test, or t[0-9]{4} followed by
".$number" to say which particular test to skip
- For an individual test suite --run could be used to specify that
only some tests should be run or that some tests should be
excluded from a run (the latter with '!' prefix).
- 'prove' harness can also run individual tests; one of more useful
options is --state, which for example would allow to run only failed
tests with --state=failed,save ... if the tests were independent.
Adding the complexity of storing a copy of the commit-graph file for
re-use in a later test is wasted energy right now, because we need to
run the steps of the test that create the repo shape with the commits
laid out as set earlier in the test. This shape changes as we test
different states of the commit-graph (exists and contains all commits,
exists and doesn't contain all commits, etc.)
I think we can solve most of the problem by separating validation tests
(which all or almost all use the same commit-graph file) and other test;
putting them in different test scripts. This means that the more
complicated case would be limited to the subset of tests.
Anyway, if the setup stages are clearly separated and clearly marked as
such, we would be able to at least manually skip tests, or manually run
only a subset of tests.
Test independence is certainly something nice to have, but as the git
testsuite is not in best shape wrt this, it is not a requirement.
I'm all for making the test suite better. In this case, I will hold that
for a later series (that is entirely focused on that feature) as I
expect we will want to discuss the correct pattern in detail.
Thanks,
-Stolee
