On Fri, Mar 05, 2010 at 01:46:34PM -0800, Adam Williamson wrote: > Ah. You're looking at it on a kind of micro level; 'how can I tell this > package has been tested?' For a package maintainer it is especially interesting, whether the own update has been tested. > Maybe it makes it clearer if I explain more clearly that that's not > exactly how I look at it, nor (I think) how the rest of QA sees it, or > what the proposal to require -testing is intended to achieve. We're > thinking more about 'the big picture', and we're specifically thinking > about - as I said before - the real brown-paper-bag, > oh-my-god-what-were-they-thinking kinds of regressions, the 'systems > don't boot any more', 'Firefox doesn't run' kinds of forehead-slappers. > What we believe is that requiring packages to go to updates-testing for > some time improves our chances of avoiding that kind of issue. Afaics, this misunderstanding is a big problem, e.g. my expectations of updates testing also differ. Maybe you can add some more information to the wiki about what QA for updates testing currently tries to ensure, what it actually ensures and plans for the future. E.g. I always noticed that there is not much karma given, but now that I wrote the script and noticed that I provided more feedback within two days than the top tester for F11 did until now, the visible test coverage was/is a lot worse than I imagined. > Obviously, the more testing gets done in updates-testing, the better. > Hopefully Till's script will help a lot with that, it's already had a > very positive response. But the initial trigger for the very first I just did a quick evaluation. There are 384 updates in F12 updates testing when I last ran fedora-easy-karma and only 108 (28%) received any comment with karma != 0. For F11 its 34/272 (12.5%). I am curious to to how these numbers have changed in a week. I hope then everyone from the QA SIG is using the script to report feedback, so it will be save to say that an update was not tested at all if it did not receive any feedback. Regards Till [1] https://admin.fedoraproject.org/updates/metrics/?release=F12 [2] https://admin.fedoraproject.org/updates/metrics/?release=F11
Attachment:
pgpMpLN7GMQwA.pgp
Description: PGP signature
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
