On Wed, 2 Dec 2020 at 18:27, Adam Williamson <adamwill@xxxxxxxxxxxxxxxxx> wrote:
On Wed, 2020-12-02 at 09:23 -0500, Ben Cotton wrote:
>
> == How To Test ==
> See QA test cases : https://fedoraproject.org/wiki/Category:CoreOS_Test_Cases
>
> We also have regular tests days, for example
> https://fedoramagazine.org/fedora-coreos-test-day/
So...yeah, that's not really enough to call something a Fedora Edition
:)
I think this has a lot of the issues we had with IoT, but turned up to
11.
We have an entire process for releasing a thing called "Fedora" which
is based around the idea that all the bits that make up a "Fedora
release" get built, tested, and signed off together.
IoT stretched this process a bit, because it's not actually built as
part of the same composes as all the other "Fedora" bits. So we had to
implement an entire parallel validation process just for IoT composes.
But at least it's built *the same way* as Fedora, so we could more or
less just split existing things into two paths and use them, which is
what we did. We now have both mainline and IoT composes and validation
events. Even there, the process wasn't perfect for Fedora 33; if you
look at the log of the Fedora 33 Go/No-Go meeting[1], which is supposed
to be where we go over the status of the bits-to-be-released in great
detail and decide whether to sign them off, there is precisely one line
about IoT:
17:58:36 <pwhalen> IoT is also covered - https://fedoraproject.org/wiki/Test_Results:Fedora-IoT_33_RC_20201020.0_General#ISO_Install
...no-one else said a word about it. So yeah, we clearly don't have
this whole "releasing from multiple streams" entirely down yet.
CoreOS is going to be the same only worse, because it's not even built
the same way as the rest of Fedora. It's not built by Pungi, we don't
get the same messages published when CoreOS builds happen (we don't get
messages published at all, IIRC), the metadata for CoreOS builds is not
in productmd[2] form like the metadata for Pungi builds, the whole
process is entirely different.
Recently messages were added when streams are updated [0][1]. I believe that other messages could be added if needed.
So to boil this down into a representative question: when we are doing
the Fedora 34 Go/No-Go meeting in ~four months' time, how do we decide
whether to release "Fedora CoreOS 34"?
Fedora CoreOS 34 does not exist, you have Fedora CoreOS stable, testing and next, each stream is released every 2 weeks.
The Go/No-Go is based on reports coming from each stream, next stream is promoted to testing and testing promoted to stable.
If any blockers are found in next or testing the content will not be promoted to stable.
The Go/No-Go is based on reports coming from each stream, next stream is promoted to testing and testing promoted to stable.
If any blockers are found in next or testing the content will not be promoted to stable.
I think it is fairly well explained here[2]
What does the question "is Fedora CoreOS 34 ready to go" even mean, in
the context of how CoreOS is built and released? What set of bits will
we be deciding to ship or not ship, and how will that have been decided
and communicated? Where will we look to find the test results and
criteria on which we would base that decision?
To maintain a fortnightly cadence there is a strong reliance on CI, every build is tested and results are inspected during the release process.
Currently a release is gated on tests running on AWS, GCP and Openstack, these tests are running on a centos-ci jenkins instance which I think cannot be access without a centos account (I might be wrong), but yeah making these tests and results more transparent could be an improvement.
Are any of these silly questions, which would indicate that our release
process is based on assumptions which need to be fundamentally re-
examined as part of this Change?
So what defines an Edition ? I think if we don't want to accept a different philosophy about release schedule and release engineering we can just close that Change proposal.
How do you consider Rawhide for example ?
All of this is stuff we could kind of handwave so long as CoreOS wasn't
an official Edition; we could *more or less* leave the CoreOS team to
decide what a CoreOS release looked like and when it would happen and
when it was good enough to ship, and so on.
So this change comes down to Can we have a Fedora Edition that has a different workflow ?
But if we're going to call it an Edition, which is one of the primary
things that defines what Fedora *is*, I don't think we can do that any
more. We need more groups to think about and decide on the answers to
questions like the above.
[1]: https://meetbot-raw.fedoraproject.org/fedora-meeting-1/2020-10-22/f33-final-go_no_go-meeting.2020-10-22-17.00.log.html
[2]: https://github.com/release-engineering/productmd
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
