On Thu, 2016-03-31 at 19:38 +0200, Giulio E. wrote: > > Hi Adam. > > I'd like to ask a question about this temporary part of release > validation testing. > > Which one of the following scenario do you recommend? > Scenario 1) Sticking with Alpha 1.7, without performing updates. Fill > in the matrices with results coming from it. > Scenario 2) Running the test cases with an up to date F24 Alpha system. > Fill in the matrices with it. > > Both of these have drawbacks: > Scenario 1) The wiki ends up with a mix of Alpha 1.7 and "updated > Alpha" results. So do the stats. > Scenario 2) If I remember right, Gnome 3.20 has been pushed stable > after the release of F24 Alpha. So, testing the version shipped with > Alpha 1.7, might not be so useful, when approaching Beta and Final test > cases. > > In my opinion, the first scenario is not a big problem. Stats are > interesting, but a proper testing looks more important and nobody will > complain if we mess up some matrices. But, if we report an already > fixed problem, we waste time. > > So... I'd rather test an up to date system. > What do you think? So far as the matrices go, I personally prefer reporting results really *for the compose in question* - it gets confusing if different people report results for different updates to the same matrix pages, you have 'bob' saying GNOME works fine and 'anna' saying it's completely broken and it becomes a bit of a mess. But I'd say for non-install-related tests it would be more valuable to test the up-to-date bits and just don't report results to the wiki - just file bugs if you find any, and nominate them as blockers if appropriate. We'll get a newer event soon enough. Always keep in mind that *ultimately* the goal of the testing is to find bugs and fix them, not check boxes in wiki pages =) And you get 'Heroes of Fedora' credit for filing bugs as well as filling in wiki boxes! This is another reason it would be nice to have a more flexible test result system than the wiki, of course. We'll get that off the todo list one day! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@xxxxxxxxxxxxxxxxxxxxxxx
