On Fri, 2016-01-29 at 08:09 -0500, Kamil Paral wrote: > > > > https://fedoraproject.org/wiki/Template:Installation_test_matrix > > > > quite a lot of the tables have 'i386' and 'x86_64' as environments. > > Especially with the Milestone column, listing i386 alongside x86_64 is > > a bit misleading if i386 is no longer blocking. I can see a few > > options: > > > > 1) just ditch the i386 columns entirely; openQA can continue testing > > it, and people can test manually if they want, but we don't bother > > tracking the results in the validation pages > > > > 2) stick an admon template at the top of the page saying 'the i386 > > tests aren't blocking', with links out to the criteria and/or the FESCo > > ticket > > > > 3) Duplicate each table which distinguishes between 'i386' and > > 'x86_64', so we have one table with just an 'i386' column and all tests > > marked Optional, and another table with the other columns and the > > appropriate milestone > I think the answer here is largely dependent on what we want to do > about OpenQA. If we didn't have OpenQA at all, I'd do #1, and I'd > also duplicate Workstation* and Server* rows in "Default boot and > install" section, gray out x86_64 and UEFI columns, and mark the rows > as optional. Therefore i686 would be handled the same way we handle > spins - there's a way to mark a critical error which prevents default > install and boot in the matrix, but that's it. > > The same solution would probably apply if we decided to drop i686 > testing from OpenQA. > > But if we want to still test i686 in OpenQA (at least on a best- > effort basis), we somewhat rely on the wiki pages for reporting. (Or > do you think that having it in OpenQA frontend is good enough?). > So if we want to keep the matrices around for that purpose, I'd > either do #3 and collapse them by default, or I'd create a separate > wiki page just for i686 and direct OpenQA results there. This way we > can still easily see what was and what wasn't tested, and tools like > testcase_stats work for it, but we the core wiki matrices are not > overflowing with non-essential stuff. But it is some work and > maintenance, and I'm not sure it's worth it. Maybe the OpenQA > frontend is just good enough? So I decided to just bite the damn bullet and do something here (I was supposed to implement something weeks ago, it's been a running joke at meetings). I mostly followed kparal's suggestion for 'if we didn't have openQA': I kept the i386 tests only for 'Default boot and install' and 'USB media' and split them into separate tables below the main tables that are collapsed by default. I renamed all 'x86' environments to 'x86_64'. This will need some changes to the openQA wiki result reporting code, I'll fix that up right away. So far as tracking the openQA i386 results goes I think the openQA web UI and the compose check emails ought to be enough. I might improve check-compose a bit to report separate pass/fail counts by arch, so it's clearer when i386 is totally busted. We still have i386 Cloud images for right now so I kept that table in the Cloud matrix, but they may go away Real Soon Now and if so I'll ditch the table. If anyone really hates this, do yell, nothing is permanent in the wiki ;) https://fedoraproject.org/w/index.php?title=Template:Installation_test_matrix&diff=441403&oldid=439868 -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@xxxxxxxxxxxxxxxxxxxxxxx
