On 14. 06. 21 18:35, Neal Gompa wrote:
On Mon, Jun 14, 2021 at 11:57 AM Miro Hrončok <mhroncok@xxxxxxxxxx> wrote:
On 14. 06. 21 17:52, Vitaly Zaitsev via devel wrote:
On 14.06.2021 15:32, Ben Cotton wrote:
Running upstream tests is mandatory.
What about tests that require network access?
From the proposed guidelines:
If a test suite exists upstream, it MUST be run in the %check section and/or
in Fedora CI. You MAY exclude specific failing tests.
You MUST NOT disable the entire testsuite or ignore the result to solve a
build failure.
As an exception, you MAY disable tests with an appropriate %if conditional
(e.g. bcond) when bootstrapping.
Assuming that *all the tests* require network access, you should run them on
Fedora CI instead.
Assuming only some of the tests require network access, you should exclude them
in %check (and possibly run them on Fedora CI instead).
This is unreasonable, since Fedora CI isn't usable across all
architectures. Unless Fedora CI is at platform parity, then we have to
have an allowance for not running tests.
Yeah, the situation wrt. platform parity is not good. However I still think
that running the network-needing tests on x86_64 only is a magnitude better
than "%%check requires internet, commented out".
That said, requiring to run tests on the Fedora CI if they cannot run in %check
is a new thing and I would not like the new Python packaging guidelines to be
blocked exactly on this. I don't event think this was the intention.
However, I have different reasons than platform parity: I don't want Python
packagers to be blocked by figuring out how to do Fedora CI, since it is
non-trivial.
Personally, I still prefer tests in %check because they are easily verifiable
e.g. in Copr when we try to measure the impact of some of our changes. Tests in
the CI are unusable for this use-case, because there is no tooling around it.
I suppose allowing to not run tests (at all) if they (all of them) require
resources not available in Koji during rpmbuild (such as internet access or
secret keys) is a reasonable compromise, however I'd like to give Petr a chance
to pitch in as well.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
