On Tue, 2010-03-09 at 17:13 -0700, Kevin Fenzi wrote: > > Some basics I'd propose as a starting point for defining acceptance > > criteria include: > > > > 1. repoclosure/conflicts - no package update can introduce broken > > deps or conflicts. I'd recommend we apply this to both > > 'updates-testing' and 'updates' (but that's detailed below) > > 2. Package sanity > > * No rpmlint failures > > I think this one should not be there. Or should be heavily filtered. > rpmlint sometimes marks things as errors that are not. +1 here. The current upstream rpmlint errors/warnings list is not what we'd want to use to automatically approve/deny updates, I'm fairly sure. =) We'd need to either get quite drastic changes upstream, or overlay our own error/warning split on the tests (which is what Mandriva does; there's an automated rpmlint run on every package submission and it rejects packages if certain tests fail, but the definition of which tests are critical is *not* the upstream one). Over time this would work fine, but at the start it may possibly introduce some absurdity due to 'grandfathered in' situations: an update may be rejected due to some lint failure which was actually present in the version of the package that's being updated anyway (it's not a _change_ in the update). I'm not sure if that's really a problem - we could just argue that the problems should be fixed and when you're pushing an update is as good a time as any to fix them - but I thought it'd be worth mentioning in advance just in case. Anyway, we should be very careful and conservative to start with, in terms of what automated tests we introduce. I'd recommend we do a week or two where we 'dry run' the system - generate lists of updates which would have been blocked, but don't actually block them - to make sure the results are sane, before we start actually blocking things. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
