On Fri, Jul 16, 2021 at 04:46:12PM +0200, Ævar Arnfjörð Bjarmason wrote:
> > and so on. That requires two new features in test-lib.sh:
> >
> > - making a GIT_TEST_RUN variable that is the opposite of GIT_TEST_SKIP
> > (instead of just the command-line --run).
> >
> > - adding GIT_TEST_{RUN,SKIP}_FROM variables to read the values from a
> > file rather than the environment (I suppose the caller could just
> > stuff the contents into the variable, but I expect that test-lib.sh
> > may want to pare down the entries that do not even apply to the
> > current script for the sake of efficiency in checking each test).
> >
> > That infrastructure would then be applicable to other cases, too. Or
> > even just useful for using another list (or no list at all) when you
> > are looking at whether other tests are leak-free or not.
>
> I've included a mechanism for whitelisting specific globs, the idea was
> not to have that be too detailed, but we'd e.g. get to the point of t00*
> or whatever passing.
>
> Anything that's a lot more granular than that is doing to suck,
> e.g. exposing teh GIT_TEST_SKIP and --run features. of specific test
> numbers, now you need to count your tests if you add one in the middle
> of one of those, and more likely you won't test under the mode and just
> see it in CI.
I think you can do the same level of skipping with GIT_TEST_SKIP,
though. My argument was just that adding a new mechanism does not make
sense when we already have one. I.e., running:
GIT_SKIP_TESTS='
t[123456789]*
t0[^0]*
t00[^016]*
t000[469]
t001[2459]
t006[0248]
' make SANITIZE=leak test
works already to do the same thing. The only thing we might want is a
nicer syntax (e.g., to allow positive and negative patterns, or to read
from a file). But that would benefit all users of GIT_SKIP_TESTS, not
just people interested in leaks.
> It also means everything works by default, you get an appropriate notice
> from prove(1), and even if you run one test manually it'll skip, but
> emit a message saying you can set the env var to force its run.
With GIT_SKIP_TESTS you obviously don't get a message saying "try
skipping this test" when it fails. :) But IMHO that is not that big a
deal. You'll get a test failure with good LSan output. If you are
working on expanding leak-checker coverage, you already know about your
options for skipping. If you're adding a new test that leaks, you might
consider fixing the leak (though not always, if it's far from code
you're touching).
-Peff
