On Fri, 05 Mar 2010 13:46:34 -0800, Adam wrote: > Ah. You're looking at it on a kind of micro level; 'how can I tell this > package has been tested?' Exactly. Because I don't like to act on assumptions. And "zero feedback" is only an indicator for "doesn't break badly", if there are N>1 testers with N>1 different h/w and s/w setups who have installed the update actually and have not rolled back without reporting a problem. This may apply to certain core packages, but _not_ to all pkgs. Not everyone runs "yum -y update" daily. Not everyone installs updates daily. It may be that there are broken dependencies in conjunction with 3rd party repos only (Audacious 2.2 test update as an example again - the bodhi ticket warned about such dependency issues, and nobody complained about them - all I know is that there are users who use Audacious, just no evidence that the test-updates are tested, too). It takes days for updates to be distributed to mirrors. A week may be nothing for that important power-user of app 'A', who would find a problem as soon as he *would* try out a test-update. Further, I hear about users who have run into problems with Fedora but haven't reported a single bug before. ABRT may help with that, but they would still need to create a bugzilla account, which is something they haven't done before and maybe won't do. Only sometimes a problem annoys them for so long that they see themselves forced to look into how to report a bug. > Maybe it makes it clearer if I explain more clearly that that's not > exactly how I look at it, nor (I think) how the rest of QA sees it, or > what the proposal to require -testing is intended to achieve. We're > thinking more about 'the big picture', and we're specifically thinking > about - as I said before - the real brown-paper-bag, > oh-my-god-what-were-they-thinking kinds of regressions, the 'systems > don't boot any more', 'Firefox doesn't run' kinds of forehead-slappers. > What we believe is that requiring packages to go to updates-testing for > some time improves our chances of avoiding that kind of issue. The key questions are still: Which [special] packages do you want to cover? CRITPATH only? Or arbitrarily enforced delays for all packages? For example, it would make sense to keep those packages in updates-testing for an extended period, which have received feedback in bodhi _before_ and which have a high bug reporting activity in bugzilla. > Obviously, the more testing gets done in updates-testing, the better. > Hopefully Till's script will help a lot with that, it's already had a > very positive response. But the initial trigger for the very first > proposal from which all this discussion sprang was wondering what we > could do to avoid the really-big-duh kind of problem. I cannot answer that. Especially not because a package that may work fine for you and other testers, may be a really-big-duh for other users. ;) This also leads to a not so funny scenario, where the big-duh has not been noticed by any tester during F-N development, but shortly after release it is found by ordinary users. When I give +1 karma, I either acknowledge only the fix for a specific bug that's linked, or I mention the type of usage, e.g. "basic daily usage" or "didn't try the new features". To not give a false impression that I may have tested everything. In general I hope that the feedback about me using the software is more helpful than zero feedback. However, it may still be that a certain feature/plugin I don't use is broken badly. That's not a guess, it has happened before and will happen again. With updates or shortly after a new Fedora release. -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
