#135: Fedora 23 Retrospective
---------------------+---------------------
Reporter: jzb | Owner:
Type: task | Status: new
Priority: normal | Milestone: Future
Component: --- | Resolution:
Keywords: meeting |
---------------------+---------------------
Comment (by kushal):
Replying to [comment:1 pbrobinson]:
> The QA process was clearly something that went terribly wrong (again)
and needs a large rethink and to be better aligned with the Fedora QA
general process.
>
> A few big ones:
> * updates in testing not getting appropriate karma (docker at least)
> * apparent critical release criteria having proper test cases and
actually in the matrix to ensure verification eg ansible [1] and root bind
mount[2].
>
> We should never get to a case we're in the 11th hour and 59th minute of
a release and discover an apparent core feature [3] is terribly broken.
>
> To be clear this isn't directed at any one person, but the whole cloud
WG. It really wasn't good enough. If it can't be tested by automation then
it should be tested manually, either way the results from any automation
should be closely checked and verified.
>
I agree that we did not do proper job in testing Cloud images for Fedora
23 on time. The autocloud project was put into production almost in the
last moment. It still needs to be monitored closely and 100s of more tests
are required to test the images properly.
We are also missing any dedicated cloud qa volunteer group from this SIG.
To start fixing the issues, I have started with few new volunteers (and I
also made a public call for help [1]), who will help us to do manual
testing of the cloud images. They will also debug any failures in the
autocloud side. I have already got ack from the Infra about getting access
to the Fedora Infra Cloud where these new volunteers will test the images.
They are also helping us to write more tests for autocloud project. A few
of those are already live in production. We are also making sure to create
new regression tests for anything we find over mailing lists/bugzilla/and
other places.
[1] https://kushaldas.in/posts/need-help-to-test-fedora-cloud-images.html
--
Ticket URL: <https://fedorahosted.org/cloud/ticket/135#comment:2>
cloud <https://fedorahosted.org/cloud>
Fedora Cloud Working Group Ticketing System
_______________________________________________
cloud mailing list
cloud@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
