On Mon, Sep 22, 2014 at 08:22:11PM -0700, Adam Williamson wrote:
> Well, I set out this morning to move the rescue mode test case from
> Alpha to Beta and somehow wound up doing this instead.
>
> Opening a release validation test page 'template', copy 'n' pasting the
> contents into a new page, and hand-editing bits until your eyes glaze
> over? That's the old thing. The new thing is this:
>
> {{subst:Validation_results|testtype=Base|release=21|milestone=Alpha|
> compose=RC1}}
>
> Create a new wiki page with just that as the contents, and you get a 21
> Alpha RC1 Base validation results page. Neato, eh?
>
> It works for any 'testtype' which has a matrix template. I ripped the
> tables out of the old templates and made them into bare templates:
>
> https://fedoraproject.org/wiki/Category:QA_test_matrix_templates
>
> So you can pass 'Base', 'Cloud', 'Desktop', 'Installation' etc as the
> testtype.
>
> This system uses two other templates, along with the matrix templates:
>
> https://fedoraproject.org/wiki/Template:Validation_results
> https://fedoraproject.org/wiki/Template:Release_validation_instructions
>
> When you substitute Validation_results into a page, it makes the page
> *transclude* the instructions template and *substitute* the appropriate
> matrix template. The upshot is that if you change
> https://fedoraproject.org/wiki/Template:Release_validation_instructions
> the changes will be reflected in any test results page (past, present
> and future), but when you edit a matrix template, the changes will only
> affect pages created later. If you need to add, remove or modify a test
> case and the change should apply to the build currently under test,
> you'll have to edit that page as well as the template page.
>
> The Release_validation_instructions template is a meld of the
> instructions from the old Install template and the newer Desktop
> template (which was cloned for Base, Server, Cloud etc etc), with some
> new improvements as well. Roshi, you might want to check the results are
> good enough for Cloud, and carefully tweak the instructions template if
> not (but bear in mind it has to remain generic, applicable to all test
> types).
>
> There's one unfortunate inconsistency here: the 'installation' results
> pages have always been called Test_Results:Fedora_(whatever)_Install,
> but the redirect page is Test_Results:Current_Installation_Test and the
> category is Category:Installation_validation_testing . I went with the
> majority vote and stuck with 'Installation' in this approach - you have
> to set testtype=Installation , not testtype=Install . We can still call
> the page _Install if we want, but it'd probably be more consistent to
> call it _Installation going forward.
>
> There's one known bug, which is that the created pages are put in the
> category Fedora_(release)_(milestone)_(compose)_Test_Results . That
> sounds right, but actually, it's meant to be _RC_Test_Results or
> _TC_Test_Results.
>
> Aside from that, the links and categories should all be 'magically'
> correct, they should not need hand-editing after page creation.
>
> There's a few things we can do there. Either we can just go with the
> more-specific categories, and remember to add them all to the higher
> level categories. Or we can drop the compose type entirely and just have
> 21_Alpha_Test_Results and 21_Beta_Test_Results and so on; I'm not sure
> we really need to split TC and RC. Or we can wait for the infra folks to
> add an extension to mediawiki which does string parsing, then I can
> parse the compose parameter to get the correct result, but they say
> that's probably going to happen for F22, not F21. Or we can simply
> hand-edit the pages after creating them, it's not so arduous. :)
>
> I have vague plans to write a little script using one or other of the
> zillion python mediawiki interfaces out there that would make creating
> the pages and handling the categories for a new TC/RC build a one-line
> operation. If I don't get distracted by something else shiny in the
> meantime!
>
> I think I lined up all the categories and updated the relevant docs -
> https://fedoraproject.org/wiki/QA:Release_validation_test_plan and
> https://fedoraproject.org/wiki/QA/SOP_Release_Validation_Test_Event are
> updated with the new procedure, and I did some related category cleanup
> along the way. The new templates all have documentation.
>
> Please let me know if you see any issues or possible improvements!
> Thanks folks :)
> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
Awesome! That makes it *way* easier to create/update the matrices. I tweaked
the cloud test matrix a bit - but yeah, it'll work great for the cloud stuff.
Thanks for putting it together!
--
// Mike
--
Fedora QA
freenode: roshi
http://roshi.fedorapeople.org
--
test mailing list
test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test
