On Tue, 2013-12-17 at 00:57 -0700, Chris Murphy wrote: > On Dec 16, 2013, at 6:33 PM, Adam Williamson <awilliam@xxxxxxxxxx> wrote: > > > Actually - it'd basically just be the 'guided installation' table from > > https://fedoraproject.org/wiki/User:Adamwill/Draft_storage_matrix#Guided_installationwithout all the other bits, basically. > > I basically get this. I expect there'll be separate matrices or pages for the archs? oh, god, handwave handwave. That's the 'matrix needs to exist in three dimensions' problem: there's just too many damn variables. What do we do if for a given test it matters what storage format you use, *and* whether you're UEFI or BIOS, *and* whether you, I dunno, pick encryption or not? The fundamental problem that makes storage validation design such a hard problem to solve is there's just too many inter-related factors. If you draw up the table to test every operation under all the different possible conditions in which you could test it, it's just unmanageable. So...I'm not sure if there would be separate pages per arch, no. But please, if you have a viable design which covers as many variables as possible without producing just too many tests for us to have a hope of covering, please suggest it! On a meta level, though, I think it's not possible for us to come up with "the perfect" storage validation test plan and have it be actually viable to accomplish: we're going to have to cut out _some_ variables. It's just a question of which combinations are the most important and practical to test, and which we have to pass on. > Question for EFI as an arch, we haven't been testing every layout > presumably because of overlap with x86_64. More or less. The current greying out of EFI tests was drawn up by me running through the table very hurriedly one afternoon in the middle of the F20 cycle, making a quick call as to what things it was most important to test separately on EFI and BIOS. It's almost certainly not perfect - please do contribute improvements / corrections to this determination. > But I'd guess the QA:Testcase_dualboot_with_windows tests for > existing GPT, and EFI System partition, rather than actually creating > them (correctly). I'll bet the code that does that is called once for > all of the guided partition schemes. So it seems like if we check the > Windows case to make sure it doesn't break Windows, we also get a "use > existing GPT and ESP" test of code; but still need one that "creates > new GPT and ESP" layout test of the code. Yes? I did that on Mac EFI, > but it's got slightly different code. Well, define 'need' :/ It's the same problem as above - too many damn paths, too much stuff that we would _like_ to cover in an ideal world. We have to somehow make a determination about what we don't cover, because we can't cover everything. > > > Do we think it's worth taking that bit out of the larger storage rework > > proposal and sticking it in the current matrix, replacing the > > appropriate existing test cases? It would be an hour or two's work for > > me, I guess. Eh, I guess I'll just draft it up and send it out and see > > what you all think. > > Why not just delete the appropriate existing test cases rather than merging them? That was what I meant by 'replacing the appropriate existing test cases' - if we like this plan we put the new matrix in and take the old test cases out. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
