On Tue, 2010-01-12 at 07:48 -0500, James Laska wrote: > On Tue, 2010-01-12 at 09:53 +0000, Adam Williamson wrote: > > On Tue, 2010-01-12 at 17:10 +0800, He Rui wrote: > > > Hi Adam, > > > > > > Nice matrix! > > > > > > I have a question. Is release level 'Alpha', 'Beta', 'Final' the same as > > > priority 'tier1', 'tier2', 'tier3'? For example, if Beta Candidate is > > > under testing, will the cases of Alpha release level be tested again? > > > > > > If all the cases should be tested on Final candidate, and many people > > > attend the tests, then both the installation and this desktop matrix > > > would be long. Then I prefer a separate page. > > > > Yes, all the tests will be performed at all stages. > > > > Note that there are some other criteria we should probably have tests > > for, which don't exactly fall into the category of installation or > > desktop (e.g. 'all services should start successfully in a default > > install'). I'm not sure where to put those. > > This topic came up yesterday in some discussion with John Poelstra and > Jesse Keating. The idea was something around an [alpha|beta|final] > release checklist. It might be a slightly different focus from what > you're talking about, but I think we could adapt it to fit. Things it > might include ... > > * All services start successfully in a default install (as you > mentioned) > * No SELinux AVC denials on initial boot or subsequent login in a > default install > * No ABRT crash notifications on initial boot or subsequent login > in a default install These two I've already written up as a 'desktop' test case and included in the matrix. > * The BetaNag has been removed from the installer > * Fedora-release package contains final version > * Kernel no longer compiled with debugging enabled > * All packages signed with the appropriate key? > * Anything about release notes? > * What else? > > Thoughts? That definitely sounds like something we should do to avoid the always-possible releng brown paper bag situations, but doesn't quite address what I'm worrying about for the QA validation tests :) > > > If anyone has some awesomely > > clever idea for combining all validation testing in a convenient way > > which isn't one gigantically long page...we're all ears :) > > We can play games with a read-only master page which transcludes all > other [[Test_Result]] content for easier test and result management. That sounds like a sensible approach - let's work on it after I get back home. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
