On Thu, 13 Feb 2025 09:32:06 +0100 Clement Verna <cverna@xxxxxxxxxxxxxxxxx> wrote: > cross posting from > https://discussion.fedoraproject.org/t/gating-fedora-updates-on-fedora-coreos-ci/144566 > > Hi all, > > Last year, the Fedora CoreOS working group implemented CI testing [1] for > Bodhi updates on a set of critical packages [2]. Automatic updates are a > key feature of Fedora CoreOS, and this testing helps us detect update > related issues early, improving Fedora’s update stability and reducing > troubleshooting time. > > While our long-term goal is to implement this CI testing in fedora-bootc > with Bodhi gating integration, there's still significant work ahead before > we can trigger fedora-bootc tests on Bodhi updates. It's worth noting that > many of the tests currently running in Fedora CoreOS CI are essentially > "image mode" tests rather than CoreOS-specific tests. Eventually, we expect > to migrate these tests to fedora-bootc. However, until that infrastructure > is ready, enabling gating on the FCOS suite provides immediate image mode > coverage for critical packages. > > Given our experience running these tests, we would like to propose making > the coreos.cosa.build-and-test a required gate for package updates in > rawhide. We've already been successfully gating packages owned by the > Fedora CoreOS working group [3], and we'd like to extend this requirement > to the broader package set defined here [4]. > > Following is the breakdown of passed vs failed builds by package on over > 400 builds, this gives package maintainers an idea of how often an update > might be gated. It is important to note that not all test failures here are > related to the software in the proposed Bodhi update since there could be > flakes; either due to the test infra environment or due to some transient > test pipeline misconfiguration. In the case where failures are not related > to updates , it would be easy to waive the test or coordinate with the > Fedora CoreOS working group to disable the test. gating based on flaky tests or flaky infra is a no-go, sorry ... You should define an "acceptable false positive" rate first (1%?, 2%?), then fix tests and infra and then think about gating. Even when half of the presented failures are not caused by the package under test, it's too much. Dan -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
