Hi Eric,
On Wed, 17 Oct 2018, Eric Sunshine wrote:
> On Wed, Oct 17, 2018 at 6:18 AM Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote:
> > I realized yesterday that the &&-chain linting we use for every single
> > test case takes a noticeable chunk of time:
> >
> > $ time ./t0006-date.sh --quiet
> > real 0m20.973s
> > $ time ./t0006-date.sh --quiet --no-chain-lint
> > real 0m13.607s
> >
> > My suspicion: it is essentially the `(exit 117)` that adds about 100ms to
> > every of those 67 test cases.
>
> The subshell chain-linter adds a 'sed' and 'grep' invocation to each test which doesn't help. (v1 of the subshell chain-linter only added a 'sed', but that changed with v2.)
>
> > With that in mind, I would like to suggest that we should start to be very
> > careful about using subshells in our test suite.
>
> You could disable the subshell chain-linter like this if you want test the (exit 117) goop in isolation:
>
> --- 8< ---
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 3f95bfda60..48323e503c 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -675,8 +675,7 @@ test_run_ () {
> trace=
> # 117 is magic because it is unlikely to match the exit
> # code of other programs
> - if $(printf '%s\n' "$1" | sed -f "$GIT_BUILD_DIR/t/chainlint.sed" | grep -q '?![A-Z][A-Z]*?!') ||
> - test "OK-117" != "$(test_eval_ "(exit 117) && $1${LF}${LF}echo OK-\$?" 3>&1)"
> + if test "OK-117" != "$(test_eval_ "(exit 117) && $1${LF}${LF}echo OK-\$?" 3>&1)"
> then
> error "bug in the test script: broken &&-chain or run-away HERE-DOC: $1"
> fi
> --- 8< ---
You're right! This is actually responsible for about five of those seven
seconds. The subshell still hurts a little, as it means that every single
of the almost 20,000 test cases we have gets slowed down by ~0.03s, which
amounts to almost 10 minutes.
This is "only" for the Windows phase of our Continuous Testing, of course.
Yet I think we can do better than this.
How difficult/involved, do you think, would it be to add a t/helper/
command for chain linting?
Ciao,
Dscho
