On Wed, 2014-07-09 at 07:52 -0400, Kamil Paral wrote: > > Hi, folks. > > > > So, we're scheduled for Alpha TC1 tomorrow. We had a nice happy > > co-operative plan where QA and the WGs would collaborate on revising the > > release validation test process for Fedora.next... > > > > ...which, well, didn't really happen. As of this morning we were nowhere > > near having a viable validation process. So I went for plan B: I spent > > today more or less pulling the entire thing out of my ass. > > Lots of work has been done. Thanks, Adam. > > When I look at all those blank cells waiting to be filled in, I'm > getting a bit dizzy :-) > > Do you think it would make sense to use the same approach as we used > in Rawhide test matrices for selected F21 matrices as well? I.e. > instead of erasing all cells every time a new TC is released, we would > use the "put date/TC number into the cell" approach? I think it would > help us to improve our coverage - currently we don't have a good > overview of which test cases were tested regularly in previous TCs and > which were not at all, because it's hard to consolidate and compare > the past results. For RCs we could use the traditional approach of a > fresh new wiki page, but for TCs the new approach could be helpful, I > think. > > Especially in small matrices like Server or Base it could work well, > but even for larger ones it might be worth the try. And it would be a > bit less depressive than to see your results "erased" with ever new > compose. > > I have modified the results template a bit (the username is now shown > as a tooltip to reduce clutter, thanks jskladan for help) and the > result could look like this: > > https://fedoraproject.org/wiki/User:Kparal/Results_Template_Alternative > > We could set up some recommendations for when to keep the older TC > results and when to overwrite (e.g. from TC1 pass -> TC2 pass), and/or > maintain the matrix manually (remove old data when needed). There are > a lot of options in this area. > > What do you think? Hah - when i was two paragraphs into your mail, I started thinking 'hmm, we could amend the Result template' :P - great minds think alike, it seems... In a way it'd be cleaner to have the image as part of the template, but then you'd have to 'pad' the entry like this: {{result|pass|adamwill||||TC1}} which would kinda trip people up (you'd have to remember the template has space for exactly three bug references, and pad correctly...recipe for disaster, I guess.) Well, we could try and get clever in the template and heuristically determine if any of the parameters after the username is a bug ID or a build and behave accordingly, but that may be gilding the lily. :P We *do* have http://testdays.qa.fedoraproject.org/testcase_stats/ , which is kind of in the 'cute nonsense hack' category but does work pretty well, to the extent that I've started using it as an entry point for testing, sometimes. We also do have an existing mechanism and policy for transferring results from one build to the next, but doing so is pretty clunky and it'd definitely be easier just to use the approach we've been using on the monthly pages. So...I guess I'm percolating this one, it seems decent but I just want to think about it a bit more. What do other folks think? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
