Hi Peff,
tl;dr let's keep an eye on adding only test cases that do not depend on
earlier test cases' output ('setup' excluded, of course).
On Sun, 14 Feb 2016, Jeff King wrote:
> In general, my opinion is that skipping arbitrary leading tests is a
> losing strategy. It's just too easy to introduce hidden dependencies,
> and not worth the programmer time to make sure each test runs in
> isolation. But others on the list may disagree.
Yes, I disagree. And you would, too, if you had to run as many tests as I
had to do by way of js/mingw-tests. I did not keep precise track of time,
but I am certain that I had to run one of those bloody tests (forgot which
one) around 100 times, each taking roughly 3 minutes to complete, and of
course, it was the *last* test case failing, and *of course* it depended
on earlier tests to run.
It gets even worse when you think about those test cases that depend on
some prereq such as SYMLINKS or POSIXPERM. In *most* cases does the
developer who adds them not even notice that these prerequisites are
required. And subsequent test cases that do *not* share those
prerequisites depend (of course!) on the previous test cases' output.
Don't get me wrong, I think it would be too much pain for little gain to
clean up our act now. But I think that we have ample evidence that it
would be a plenty good idea to try pretty hard to avoid adding to the
pile.
Ciao,
Dscho
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html