Hi, folks. So, we're scheduled for Alpha TC1 tomorrow. We had a nice happy co-operative plan where QA and the WGs would collaborate on revising the release validation test process for Fedora.next... ...which, well, didn't really happen. As of this morning we were nowhere near having a viable validation process. So I went for plan B: I spent today more or less pulling the entire thing out of my ass. It's a bit rough around the edges, but I think we more or less have something workable now. I have skipped the draft stage for a lot of documents just in the interest of having something vaguely workable in time for TC1; of course, the pages can be revised as much as necessary as we work with them. It's a bit hard to remember everything I've done, but we now have a draft Alpha Release Criteria page which should cover all release-blocking media, which for now I'm assuming includes the Workstation live media, Server minimal and offline install media, Cloud and ARM disk images, and possibly some kind of generic network install image. That draft is at https://fedoraproject.org/wiki/User:Adamwill/Draft_F21_Alpha_criteria and is based on the stuff from https://fedoraproject.org/wiki/User:Adamwill/Draft_server_release_criteria , https://fedoraproject.org/wiki/User:Adamwill/Draft_workstation_release_criteria and https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Cloud_Alpha_Release_Criteria , plus some adjustments to the templates that handle the preamble. We have a new validation matrix, for Server product-specific tests: https://fedoraproject.org/wiki/QA:Server_validation_results_template The Desktop matrix has been adjusted to cover - not quite elegantly, but at least cover - both the Workstation product and the KDE spin: https://fedoraproject.org/wiki/QA:Desktop_validation_results_template The Base matrix has been extended to add a couple of new test cases that came out of the Product criteria drafting process, but actually aren't really product specific, and has had its columns adjusted to be product-y: https://fedoraproject.org/wiki/QA:Base_validation_results_template The installation matrix has similarly had a couple of new criteria wedged in, but much more importantly, I ripped the netinst, DVD and live image 'sanity test' sections and replaced them with an ARM-style table where a single 'generic' test case is run against several images on several platforms - that's the "Default boot and install" section, so please cast an eye over it: https://fedoraproject.org/wiki/QA:Fedora_21_Install_Results_Template and I made a small change to the release validation test event SOP to list the server matrix as one to create: https://fedoraproject.org/wiki/QA/SOP_Release_Validation_Test_Event Here are links to all (I think) of the new test cases I had to write as I went along: https://fedoraproject.org/wiki/QA:Testcase_Boot_default_install https://fedoraproject.org/wiki/QA:Testcase_kickstart_user_creation https://fedoraproject.org/wiki/QA:Testcase_base_service_manipulation https://fedoraproject.org/wiki/QA:Testcase_base_selinux https://fedoraproject.org/wiki/QA:Testcase_Server_firewall_default https://fedoraproject.org/wiki/QA:Testcase_kickstart_firewall https://fedoraproject.org/wiki/QA:Testcase_Server_cockpit_default The following test cases already existed, but are newly included in the release validation process (they were written for test days): https://fedoraproject.org/wiki/QA:Testcase_FreeIPA_realmd_join https://fedoraproject.org/wiki/QA:Testcase_realmd_join_kickstart https://fedoraproject.org/wiki/QA:Testcase_realmd_join_server There are still quite a few i's to dot and t's to cross. There are some release criteria and test cases that explicitly reference 'the DVD' image that will need to be adjusted. We need to apply the 'associated release criterion' template to all the new test cases, and probably clean up some categorizations. Various other process documentation pages may need to be updated, we'll have to check through all of them. But I think now we at least have the broad strokes of what's needed for .next validation testing. All feedback on the above changes is of course welcome! Please do cast your eye over and point out anything I missed, anything that looks silly, any possible improvements and so on. Remember, though, this is really *test process* design: we're not actually doing product design here, if you think there are issues with the Fedora.next changes themselves, that goes to the WGs or FESCo. As far as this work is concerned, we're just trying to test the new Products as they're designed (all of the above is based off of the Product PRDs and tech specs). I'll take time tomorrow to do some polishing, and of course look at any and all feedback on the stuff I did today. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
