Hi Zbyszek,
I'm looking for a solution which doesn't just skip the installability tests altogether.
On PRs with zuul or FedoraCI automation, the same instability tests that are done for Bodhi are performed. But what would help is to make these tests as required to pass unless they are manually waved. Manually that can be done by setting `gating.yaml`. There was some discussion on making some of these tests as gating by default.
Another issue specific to installability is that the issue is often further down the stream, particularly with the SELinux test. Definitely these need to be tracked down and fixed.
A second problem is that when the tests fail, it's just soooo hard to find out why they failed.From the bodhi status page, one has to go to the Jenkins status page, guess that it's useful to look at Console Output, scroll over a few pages of incomrehensible JSON gibberish, guess that it's worth clicking on Testing Farm Artifacts URL, click that, scroll pages of output to see "guest setup failed: Test environment installation failed: Install packages".
Weird, when the test is finished, you should have only the final
testing-farm results page. Here's an example [1]. Maybe in your
case it encountered an internal failure?
[1]: https://bodhi.fedoraproject.org/updates/FEDORA-2024-57f489c90d
-- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
