On 11/15/23 22:14, Mark Brown wrote:
On Wed, Nov 15, 2023 at 07:43:49PM +0200, Nikolai Kondrashov wrote:
Introduce a new tag, 'Tested-with:', documented in the
Documentation/process/submitting-patches.rst file. The tag is expected
to reference the documented test suites, similarly to the 'V:' field,
and to certify that the submitter executed the test suite on the change,
and that it passed.
This doesn't feel like it fits so well with git based flows - generally
the tags end up in git one way or another so there'll be a strong
tendency for this to end up getting added for one version and then
carried forward to the next version. The way the tooling is at present
it doesn't really feel like there's a good point at which to insert the
tag.
I'm not sure exactly what'd be better though.
Yeah, I agree that's a bit of a problem. One that only automated
tools/testing/CI could fully solve. Cough, git forges, cough.
OTOH, once you managed to run an automated suite once, it's much easier to do
it again, and most of the time developers *want* their code to work and pass
the tests (it's much easier than manual testing after all). So it's likely
they will keep running them for new revisions, even though they might not
notice they simply reused the previously-added Tested-with: tag.
Still, one way to make this better could be requiring a URL pointing at test
results to follow the test suite name in the Tested-with: tag. Then the
maintainer could check that they're indeed fresh.
This would be however getting way ahead of ourselves and, with the current
(average) state of testing infra, hard to do. Perhaps sometime later.
For now, I think this could already go a long way towards having more (and
better) testing.
Nick
