Hi folks! So it's been an open question exactly how we would do release validation for the Modular Server release. For Beta, we wound up trying to co-ordinate it via mailing list and IRC; I thought doing it this way would be sufficient and save time trying to set up a parallel stream of wiki validation events. However, I don't think it really turned out that well; we didn't have a great overview of test coverage at the Beta go/no-go meeting, and just wound up shipping it more or less on a handwave. And people keep asking about where they can find and report results for the Modular Server composes. So, I went ahead and spent the last couple of days enhancing all the release validation bits to handle having release validation events for Modular composes, in a way which hopefully won't screw anything up. The way it *should* work, if I got it all right, is this: The fedmsg consumer which automatically creates release validation events will now also create events for Fedora-Modular 27 composes, following the exact same rules it follows for creating regular validation events. I have also created an initial Modular validation event, for the Beta-1.5 compose we signed off as the Modular Server Beta. And I have set things up so there's a new set of 'Current' redirect pages for the Modular events, with 'Current_Modular' in their name: https://fedoraproject.org/wiki/Test_Results:Current_Modular_Summary https://fedoraproject.org/wiki/Test_Results:Current_Modular_Installation_Test https://fedoraproject.org/wiki/Test_Results:Current_Modular_Server_Test https://fedoraproject.org/wiki/Test_Results:Current_Modular_Base_Test At present those links will redirect you to the Beta 1.5 pages. You'll notice the page names follow the same basic scheme as for non-modular events, just with "Fedora Modular" replacing "Fedora". This similarly applies to all the category names, so there is now a "Fedora Modular 27 Beta Test Results" category and so on. You may note the contents of the Installation and Base results pages are a bit different, effectively I've cut them down to the bits that are relevant to a compose which is intended to deliver only Server bits. This is achieved by using separate matrix template pages: https://fedoraproject.org/wiki/Template:Base_modular_test_matrix https://fedoraproject.org/wiki/Template:Installation_modular_test_matrix I'm usually the only one who edits the matrix templates anyway, but thought I'd note it just in case. Naturally, if you want to change the result pages for modular events, you edit the modular template pages, and vice versa. Server_modular_test_matrix exists but is just a redirect to Server_test_matrix , as we don't want those tables to differ at all. For Modular events, only Installation, Server and Base pages are created; Desktop and Cloud are not needed or relevant. The consumer should create a new event for a nightly compose as soon as one appears that has significant package differences from the Beta 1.5 compose. Events for modular composes should be announced by email just as non-modular events are; the text of the email is of course adjusted slightly to distinguish between modular and non-modular events. openQA results should get forwarded to the wiki just as they do for non-modular composes (I have forwarded the results for Beta 1.5 to the wiki to test the mechanism, for future composes/events it should happen automatically). All the relval sub-commands now support operating on modular composes/events. So you can use relval to create modular events (though, again, normally the fedmsg consumer will do this for us automatically, no-one needs to do it manually), report results for modular composes, run the image size checks on modular composes, and do the user-stats and testcase-stats analysis for modular composes. For each sub-command, a new argument `--modular` is available which tells it to operate on a modular compose/event/set of events. For report- results, you can run just `relval report-results --modular` to report results for the current Modular validation event. You can find testcase_stats for 27 Modular at https://www.happyassassin.net/testcase_stats/27modular/ . This link will be included in announcement emails for Modular validation events. Updates are available for EPEL 7, Fedora 25, Fedora 26 and Fedora 27 containing the updated relval, python-wikitcms and fedfind packages. I've just submitted these, so they should reach updates-testing shortly. If you want to use relval to report results for modular composes, you'll want to update to the new versions (fedfind 3.8.0, python-wikitcms 2.2.0, relval 2.2.0). Ultimately, for the Final phase of the Fedora 27 Modular Server release, we should be able to do release validation just as we do for regular releases. For now I've implemented all of this with two assumptions: we're only going to have this split 'regular / modular server' schedule for F27, and the modular composes won't have any bits we care about besides server bits. If either of those assumptions turns out to be wrong, we'll have to revisit this again. Please do drop me a line if you spot anything wrong/weird/broken- looking in any of this modular handling. Thanks! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
