On Wed, Dec 2, 2020 at 12:19 PM Adam Williamson <adamwill@xxxxxxxxxxxxxxxxx> wrote: > > On Wed, 2020-12-02 at 09:23 -0500, Ben Cotton wrote: > > > > == How To Test == > > See QA test cases : https://fedoraproject.org/wiki/Category:CoreOS_Test_Cases > > > > We also have regular tests days, for example > > https://fedoramagazine.org/fedora-coreos-test-day/ > > So...yeah, that's not really enough to call something a Fedora Edition > :) > > I think this has a lot of the issues we had with IoT, but turned up to > 11. > > We have an entire process for releasing a thing called "Fedora" which > is based around the idea that all the bits that make up a "Fedora > release" get built, tested, and signed off together. > > IoT stretched this process a bit, because it's not actually built as > part of the same composes as all the other "Fedora" bits. So we had to > implement an entire parallel validation process just for IoT composes. > But at least it's built *the same way* as Fedora, so we could more or > less just split existing things into two paths and use them, which is > what we did. We now have both mainline and IoT composes and validation > events. Even there, the process wasn't perfect for Fedora 33; if you > look at the log of the Fedora 33 Go/No-Go meeting[1], which is supposed > to be where we go over the status of the bits-to-be-released in great > detail and decide whether to sign them off, there is precisely one line > about IoT: > > 17:58:36 <pwhalen> IoT is also covered - https://fedoraproject.org/wiki/Test_Results:Fedora-IoT_33_RC_20201020.0_General#ISO_Install > > ...no-one else said a word about it. So yeah, we clearly don't have > this whole "releasing from multiple streams" entirely down yet. > > CoreOS is going to be the same only worse, because it's not even built > the same way as the rest of Fedora. It's not built by Pungi, we don't > get the same messages published when CoreOS builds happen (we don't get > messages published at all, IIRC), the metadata for CoreOS builds is not > in productmd[2] form like the metadata for Pungi builds, the whole > process is entirely different. > > So to boil this down into a representative question: when we are doing > the Fedora 34 Go/No-Go meeting in ~four months' time, how do we decide > whether to release "Fedora CoreOS 34"? > > What does the question "is Fedora CoreOS 34 ready to go" even mean, in > the context of how CoreOS is built and released? What set of bits will > we be deciding to ship or not ship, and how will that have been decided > and communicated? Where will we look to find the test results and > criteria on which we would base that decision? > > Are any of these silly questions, which would indicate that our release > process is based on assumptions which need to be fundamentally re- > examined as part of this Change? > > All of this is stuff we could kind of handwave so long as CoreOS wasn't > an official Edition; we could *more or less* leave the CoreOS team to > decide what a CoreOS release looked like and when it would happen and > when it was good enough to ship, and so on. > > But if we're going to call it an Edition, which is one of the primary > things that defines what Fedora *is*, I don't think we can do that any > more. We need more groups to think about and decide on the answers to > questions like the above. > > [1]: https://meetbot-raw.fedoraproject.org/fedora-meeting-1/2020-10-22/f33-final-go_no_go-meeting.2020-10-22-17.00.log.html > [2]: https://github.com/release-engineering/productmd This is a fair point. I've personally been very annoyed about how little Fedora CoreOS integrates with the rest of the Fedora Project. One very broken consequence of Fedora CoreOS working this way is that they basically *don't* participate in Changes discussions and just do things differently without warning or documentation. This has led to problems when Fedora CoreOS differs from the rest of Fedora in core, fundamental ways. This has even happened unintentionally, where Fedora CoreOS accidentally deviated from Fedora (and RHEL CoreOS!) by using the wrong variant of iptables: https://github.com/coreos/fedora-coreos-tracker/issues/676 Additionally, to the point about their tooling: there's been a bunch of effort to plug OSBuild based image builds into our normal infrastructure, and given that even Fedora IoT Edition can successfully be produced through that pipeline, I would think that we should have Fedora CoreOS transition to it. There's a lot more effort going around OSBuild within the project as a whole, and it's much more likely that we'll be able to incorporate that into the general infrastructure in a way that makes it traceable and actionable within and outside the project. I would personally rather see Fedora CoreOS pulled *back* into the fold more as an Edtion. -- 真実はいつも一つ！/ Always, there's only one truth! _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
