----- Original Message ----- > From: "Adam Williamson" <adamwill@xxxxxxxxxxxxxxxxx> > To: "For testing and quality assurance of Fedora releases" <test@xxxxxxxxxxxxxxxxxxxxxxx> > Sent: Thursday, April 12, 2018 3:57:33 AM > Subject: Re: [Proposal] Test Day Guidelines > > On Wed, 2018-04-11 at 03:23 -0400, Sumantro Mukherjee wrote: > > Hey All, > > > > I would like to make a proposal for updating and fleshing out some of test > > day guideline. > > > > To start with the wording on the Top of Test Day page: > > If you come to this page before or after the test day is completed, > > your testing is still valuable, and you can use the information on > > this page to test, file any bugs you find at Bugzilla, and add your > > results to the results section. If this page is more than a month old > > when you arrive here, please check the current schedule and see if a > > similar but more recent Test Day is planned or has already happened. > > > # Proposal 1: We would also like people to *not* do testing if we are > > testing something which are compose sensitive and post-test day > > chasing and finding the same bugs in the compose and filing dups > > won't make much sense. So, it will be best to reduce the time frame > > from a month to a week. > > Also, check the current schedule for something fresher and ready > > requiring testing love. > > I picked 'a month' more or less arbitrarily, 'a week' is certainly an > OK choice too. Note that the template really is just a *template*, and > can (and should!) be adapted to each event as appropriate. So if *some > specific event* is very time sensitive, this text could be changed as > appropriate - to say 'a day' or whatever. True. I will keep this in mind from next time :) > > > # Proposal 2: We would love to see more coverage on the test cases, > > let's take Modularity and as an example, Contributors found bugs in > > Rpi 3 (ARM) which weren't the case with x86_64, Profile field in the > > test day results > > page becomes very important and crucial. Maybe, we should avoid > > putting "Fedora-Server-dvd-xxxxxyyyyyyz.n.0.iso" which doesn't give > > much info about the testing environment or how it was executed. We > > might > > want to flesh out the importance of that field and it will be very > > helpful in coming future. > > I'm actually not quite sure what you're referring to here, or where the > "Fedora-Server-dvd-xxxxxyyyyyyz.n.0.iso" example comes from. Could you > provide a bit more context? The context where it stems from was, if you have a look at http://testdays.fedorainfracloud.org/events/38 we have mostly all testers, mentioning Fedora 28 Server on (KVM,ARM,x86_64,VMware,ppc64le) which gives us a bit more idea if there is a failure. Just writing Fedora server dvd gives us absolutely no idea as to how the test were executed. In one certain case during Modularity test day, a certain issue just came up which was specific to ARM ... the profile also lets us know about the coverage across archs and various virtualizations. I felt, we should, flesh out a bit more about the importance of the same. > > In general I'd certainly say that, again, the way in which result > reporting is set up should be carefully considered for each individual > test day; the template and SOP are only guides. You can set up a result > table in many different ways to capture results from different > combinations of environment factors... yes, thats something I will look at :) > > > # Proposal 3: Test Days have a specific team which takes on the task > > of curating the #fedora-test-day channel and these are mostly the > > people who are working on the feature being tested or developed for a > > significant amount of time. > > This is done to ensure that all the contributors/participants can pin > > point folks who are playing the role they are designed for.It's in > > the best interest of contributors that they shouldn't edit this > > section as they misleading if done without proper knowledge. > > Are you saying there should be a specifically-identified 'team' for > each individual test day, or an ongoing permanent 'team' for test days > as a whole? Either could potentially work, but I think the proposal > could do with a few more details... No, I am good with either, I am strictly against people editing the organizing list. The team is defined and kept there to make sure the contributors know where to come and whom to grab hold of , when something is breaking. Randomly editing the section renders to pupose moot. Context: [http://fedoraproject.org/wiki/Test_Day:2018-02-22_Kernel_4.15_Test_Day] [http://fedoraproject.org/wiki/Test_Day:2018-04-10_Add-On_Modularity_Test_Day] > -- > Adam Williamson > Fedora QA Community Monkey > IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net > http://www.happyassassin.net > _______________________________________________ > test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx > _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
