On Tue, Jan 29, 2019 at 06:56:08AM -0800, Derrick Stolee via GitGitGadget wrote:
> When running the test suite for code coverage using
> 'make coverage-test', a single test failure stops the
> test suite from completing. This leads to significant
> undercounting of covered blocks.
>
> Add two new targets to the Makefile:
>
> * 'prove' runs the test suite using 'prove'.
>
> * 'coverage-prove' compiles the source using the
> coverage flags, then runs the test suite using
> 'prove'.
>
> These targets are modeled after the 'test' and
> 'coverage-test' targets.
I think the cover letter would be a better commit message.
> Signed-off-by: Derrick Stolee <dstolee@xxxxxxxxxxxxx>
> ---
> Makefile | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index 1a44c811aa..ec886635ae 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -2665,6 +2665,9 @@ export TEST_NO_MALLOC_CHECK
> test: all
> $(MAKE) -C t/ all
>
> +prove: all
> + $(MAKE) -C t/ prove
> +
You don't need this 'prove' target in the "main" Makefile, because
'make test' will run the test suite using DEFAULT_TEST_TARGET anyway.
> perf: all
> $(MAKE) -C t/perf/ all
>
> @@ -3077,6 +3080,10 @@ coverage-test: coverage-clean-results coverage-compile
> $(MAKE) CFLAGS="$(COVERAGE_CFLAGS)" LDFLAGS="$(COVERAGE_LDFLAGS)" \
> DEFAULT_TEST_TARGET=test -j1 test
>
> +coverage-prove: coverage-clean-results coverage-compile
> + $(MAKE) CFLAGS="$(COVERAGE_CFLAGS)" LDFLAGS="$(COVERAGE_LDFLAGS)" \
> + DEFAULT_TEST_TARGET=prove -j1 prove
First I was wondering why do you need a dedicated 'coverage-prove'
target, instead of letting DEFAULT_TEST_TARGET from the environment or
from 'config.mak' do its thing. But then I noticed in the hunk
context, that, for some reason, the 'coverage-test' target hardcoded
'DEFAULT_TEST_TARGET=test -j1'. Then I was wondering why would it
want to do that, and stumbled upon commit c14cc77c11:
coverage: set DEFAULT_TEST_TARGET to avoid using prove
If the user sets DEFAULT_TEST_TARGET=prove in his config.mak, that
carries over into the coverage tests. Which is really bad if he also
sets GIT_PROVE_OPTS=-j<..> as that completely breaks the coverage
runs.
Instead of attempting to mess with the GIT_PROVE_OPTS, just force the
test target to 'test' so that we run under make, like we intended all
along.
I'm afraid that this issue would badly affect 'coverage-prove' as well
(I didn't try). Or if doesn't (anymore?), then that should be
mentioned in the commit message, and then perhaps it's time to remove
that '-j1' from the 'coverage-test' target as well.
