On Wed, 2009-10-14 at 20:31 +0530, Rahul Sundaram wrote: > On 10/14/2009 08:23 PM, Jesse Keating wrote: > > > And that's a people problem more than a process problem. If nobody > > tests it in updates-testing, then how is the maintainer to know that it > > is problematic? Certainly not solvable with even more repos for testing > > content... > > 3 people give positive feedback and the update is automatically pushed > from updates-testing to updates despite atleast one feedback to the > contrary at > > https://admin.fedoraproject.org/updates/F11/FEDORA-2009-9911?_csrf_token=b77b748e49c5311eb85031331cb2f6474028d615 > > The UI changes certainly would be visible without any user feedback. The > buttons getting removed from the toolbar as well as smart folders were > immediately visible within minutes. Anyone with significant amount of > email would probably run into the indexing issue soon as well. > > Note that the update indicates it is a security issue but doesnt explain > what the security fix is nor does it indicate what other major changes > are there. No notes has been entered to assist the testers. I don't > think the onus can be placed entirely on potential testers to provide > feedback within a week. That is just finger pointing and doesn't help > address such problems or even mitigate it. It's worth looking in more detail at exactly what the feedback was. Here's some of the feedback which was marked positive, i.e. +1: "loving the new search stuff" "Works for me, is it intended behaviour that several buttons including "Delete" disappear off the main (top) toolbar?" without those two bits of feedback - which noted what were later identified as problems with the update, but nevertheless rated it positively - it wouldn't have been pushed. it only ever hit exactly +3, never higher. without those comments, it would have hit a max of +1. so I disagree with the notion that bodhi / updates-testing are useless ("fat lot of good"), and agree with Jesse that the evidence doesn't support the conclusion that the best fix is to throw more repositories at the problem. I'd agree with Jesse's point that it would've been best for the maintainers to disable the +3-automatic-push for this update, though hindsight is always 20-20. perhaps we (QA / rel-eng) need to give more specific advice about when to use it. My perspective would be that automatic-push should only be used when you're making a small-scale update which fixes one or two specific issues and does not change behaviour in other ways. It should not be used for a big update like this which rolls up many 'fixes' and things which can't strictly be described as fixes, because you're going to get a situation like this where it's hard for a simple +1 / -1 from individual testers to judge the update. We might also look at providing better instructions for people providing feedback on exactly when to use the +1, -1 and 0 options. I'll try and find some time to look at that. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- fedora-devel-list mailing list fedora-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-devel-list
