On 2/13/25 3:32 AM, Clement Verna wrote: > Given our experience running these tests, we would like to propose making the coreos.cosa.build-and-testa required gate for package updates in rawhide. We've already been successfully gating packages owned by the Fedora CoreOS working group [3], and we'd like to extend this requirement to the broader package set defined here [4]. Resurrecting this thread as a recent instance I think is a good example of what we're trying to avoid with this proposal. First I'd like to say that this email is in no way a criticism of the GRUB2 folks. In fact they've been great to work with in the various issues we've found over the years. This is just an example of a typical disruption that occurs that we think gating would help with. Last Wednesday there was a grub2-2.12-24.fc43 update [1] that came in that completely broke booting in CoreOS. For whatever reason this same update didn't break testing in OpenQA of other editions, but we knew immediately [2] that it broke us and reported negative karma on the update [3]. Despite this we were powerless to prevent the update from flowing in without somehow making contact with the maintainers. So our `rawhide` builds were just doomed to fail, right? Not exactly. We've been dealing with this problem for a long time, and we often get criticism for doing this (though I don't really know given the circumstances described above why), but we can pin or fast-track builds sometimes to workaround problems like this. We *need* this, because we feel strongly that we shouldn't ship (register it in our official build history and push to a place where people can download it) any build without it passing tests. This means that the history of our rawhide stream is essentially known good (passing tests) versions of rawhide. If you can download a rawide build of CoreOS, you know it's going to boot (at least for the platforms we test). So here we are. We know `rawhide` is going to break for us. Not much we can do about it on the official Fedora side of things. So we go to work. 1. Open an issue in our issue tracker [4] 2. Open a PR to pin grub2 on the older (passing tests) version [5] 3. Open a bug to report the issue to the maintainers [6] 4. Collaborate with the maintainers to investigate [7] 5. Wait for a fix 6. Unpin grub2 a week later so we can follow rawhide again [8] Needless to say this is a lot of work. Now, all of this work wouldn't just magically go away if we had gating, but it would be shifted and would be more of a collaboration between the buggy software and the CoreOS team. Today it's on us to convince people the tests we are running are really issues with their software. Sometimes we do need to adjust our tests OR we were somehow using the software in a way that was unintended and we adjust. But for the most part if the software worked yesterday, a regression is considered something that we (collectively) want to fix. [1] https://bodhi.fedoraproject.org/updates/FEDORA-2025-d929030c88 [2] https://matrix.to/#/!QJyaccWPAXLYZLCiRL:fedora.im/$1uIMeAzEK9LzKA5cG_WH9I1WtqbUiLuWCtFp7tR3Bl8?via=fedora.im&via=matrix.org&via=fedoraproject.org [3] https://bodhi.fedoraproject.org/updates/FEDORA-2025-d929030c88#comment-3967537 [4] https://github.com/coreos/fedora-coreos-tracker/issues/1878 [5] https://github.com/coreos/fedora-coreos-config/pull/3369 [6] https://bugzilla.redhat.com/show_bug.cgi?id=2346804 [7] https://bugzilla.redhat.com/show_bug.cgi?id=2346804#c6 [8] https://github.com/coreos/fedora-coreos-config/pull/3382 -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
