Hi Junio,
On Wed, 22 Jan 2020, Junio C Hamano wrote:
> SZEDER Gábor <szeder.dev@xxxxxxxxx> writes:
>
> > On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> >> * yz/p4-py3 (2020-01-15) 14 commits
> >> - ci: also run linux-gcc pipeline with python3.5 environment
> >
> > I still think that this last patch needs to be reworked before this
> > series is merged any further.
> >
> > The only Python script we have is 'git p4', so the Python version is
> > only relevant for 'git p4' tests ('t98*'), while the rest of Git and
> > the test suite couldn't care less [1]. This patch, however, not only
> > builds Git and runs the full test suite for each of the two Python
> > versions, but, worse, runs the full test suite _twice_ for each, first
> > as a "regular" test run and then again with all the GIT_TEST_* knobs
> > enabled. Consequently, it adds ~50mins to every build's runtime.
> >
> > That's just too wasteful.
>
> Thanks for a reminder. Yes, I do recall you raised the above point
> and I agree with the assessment.
>
> What's the ideal endgame wrt the tests? Build with Py$N and run
> full test suite once, and run full test suite again with the unusual
> knobs enabled, which is what is done without this series, plus build
> with Py(5-$N) and run and run only t98?? tests?
Should we declare `t98xx` to be the namespace for the Python-based
scripts, or alternatively declare that we won't ever include another
Python script but `git-p4`?
But yes, I think that we should probably "tack on" the Python 3.x tests to
the `linux-gcc` job.
Or maybe finally split this job into three: one job that does what
`linux-gcc` suggests, a `linux-gcc-knobs` one that sets all those `GIT_*`
variables, and a python3x one that only runs t98*.sh.
The reason to split it off is this: on rare occasion, I have to restart
the `linux-gcc` job because _one_ of those `git-p4` tests failed due to
some reason or other, probably timing-related, I did not have time to
investigate this. Having to re-run the entire test suite, twice, just to
work around those flaky tests is rather wasteful.
That would actually be my prereference. If people agree, I will revive
https://github.com/gitgitgadget/git/pull/266.
Ciao,
Dscho
