On Tue, 2014-05-27 at 07:58 -0400, Matthew Miller wrote: > On Fri, May 23, 2014 at 06:02:07PM -0700, Adam Williamson wrote: > > Hi, folks! So, I quickly bashed out that draft F21 Test Plan I've been > > threatening to write for the last month or so. > > https://fedoraproject.org/wiki/User:Adamwill/Draft_Fedora_21_test_plan > > So, what's the idea here? > > Psshh -- so much for your claim of not doing any real work. This looks > great! I'll put discussing it on our next Cloud WG meeting. Awesome, thanks. > > * There are several practical implications from the test plan - just > > Work We Need To Do. Most obviously, we need to draw up release criteria > > and supporting test cases for the Fedora.next Products. We also will > > need to adjust the > > Mike has some drafted up for cloud at > <https://fedoraproject.org/wiki/User:Roshi/QA/Cloud_Docs/Test_Plan#Test_Pass.2FFail_Criteria>. > It'll need a bit more by way of specifics. hah, I missed those somehow. That's a good start. My mental model was actually going to be that we'd keep the 'generic' release criteria as something shared between all Products and then just have additive Product-specific release criteria pages (they could also amend the base criteria slightly if necessary, as long as the delta didn't get too crazy), but the approach of having multiple 'complete' sets of release criteria may work out better, we'll have to think it through. I'm planning to kick off some threads about this work this week. > > * Responsibilities! Particularly, in this *draft* Test Plan, I've > > suggested that creating the Product-specific criteria and test cases > > should be the responsibility of the relevant Working Groups, with > > assistance from the QA team. They would also be jointly responsible, > > This seems like the right way to me, too. Cool, glad everyone seems to be agreeing on that. > > for Fedora 21 testing *overall* - i.e. it's the QA team's responsibility > > to make sure the WGs do the stuff assigned to them in the plan. If that > > makes sense. Discussion welcome! > > Yeah. If something isn't working, do you think the best path is QA->WG > directly, or should it be QA says something to FESCo, FESCo works with WG? As with all the others who've weighed in so far, I agree that the optimal path is just for us all to talk to each other. I generally prefer workflows that assume everyone's happy to work positively together to a consensus solution over ones that assume everything will be oppositional and must therefore go through a higher authority. So, yep, IMO we should just talk to each other, and FESCo should only be invoked to resolve irreconcilable disputes (of which we hope there won't be any). > Since parts of this are new to all of us, and other parts new to a lot of > us, maybe it'd be good to put suggested/reasonable deadlines on some parts > of the responsibilities? That helps put in focus what needs to be done. Indeed, I didn't put it in the plan but it would obviously be helpful. We should really aim to have the process nailed down and at least the Alpha criteria and test cases (and any required automated testing stuff) ready by the date of Alpha TC1. Ideally we'd have Beta and Final criteria and test cases prepared at that point too, but we _could_ delay those until those actual milestones if we have to. The actual release process is heavily tied into the release criteria, validation testing, and blocker bug stuff; if we don't have that in place it's very difficult to work through a milestone release in any meaningful way which isn't just "let's chuck something over the wall and hope for the best". Ideally I'd like to work fairly rapidly and get, say, the initial release criteria ready by the end of next week, and then get the required test cases and matrices written between then and branching (which is 'no earlier than July 8' on current schedule). -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
