On Wed, 2009-11-25 at 14:29 -0500, James Laska wrote: > Greetings folks, > > Whether you call it a post-mortem, retrospective or lessons learned ... > the end result is the same. I'd like to collect thoughts on how > good/bad of a job the QA group did in planning and testing the Fedora 12 > release. In keeping with the release-wide retrospective from Fedora 11 > [1], feel free to share any wishlist items as well. > > I've started the discussion on the wiki at > https://fedoraproject.org/wiki/Fedora_12_QA_Retrospective. > > Adding your thoughts is easy ... > * Edit the wiki directly (instructions provided for ~anonymous > feedback) > * Or, reply to this mail (I'll collect feedback and add to the > wiki) > > Over the next week, I plan to organize any feedback and discuss the > highlights during an upcoming QA team meeting. The goal will be to > prioritize the pain points and use as a basis for defining objectives > for Fedora 13. > > Thanks for your feedback! > James 1) in the end, we focused heavily on just three component areas for testing: anaconda, kernel, and X.org. This is primarily a function of the fact that these are the most vital bits; it feels like we're still at the point of doing 'let's make sure it's not totally broken' testing than 'let's make sure it's really good' testing. We didn't do stuff like making sure the desktop was polished. 2) there was clearly a lot of uncertainty about RAID issues; it's something we obviously don't as a team test well enough (and some of us personally don't understand enough :>). In the end there didn't turn out to be any horrible issues, but the confusion was evident, and we did miss the Intel BIOS RAID stuff-ups. For F13 we should have better RAID testing both in Test Days and in pre-release test cycles. 3) We weren't completely on top of X.org bugs for this release. The ones that wound up getting promoted to release blocker level were kind of an arbitrary selection. I think we got nouveau mostly right as I had a reasonable grip on nouveau triage, but we just had too few people to triage server / intel / ati bugs during the cycle, so when we hit beta / RC stage, we didn't have the whole bug set well enough triaged to be able to be sure we picked the most important bugs as blockers. For F13 we should stay on top of triage better so we can do blocker identification accurately. happily, matej is more active on X triage again now and we have some more assistance from Chris Campbell (thank you Chris!) further volunteers would be great. I will try to stay on top of nouveau, again. 4) we have the big security thing to deal with. I did start a thread on -devel about that. 5) test days went well again. it was nice to see how many 'independent' test days there were. that's what i've got so far :) -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- fedora-test-list mailing list fedora-test-list@xxxxxxxxxx To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
