On Thu, Sep 8, 2011 at 12:54 PM, Nils Philippsen <nils@xxxxxxxxxx> wrote: > On Wed, 2011-09-07 at 18:02 -0700, Adam Williamson wrote: >> On Wed, 2011-09-07 at 18:47 -0600, Kevin Fenzi wrote: >> > * As a maintainer it's easy to have a env that gets out of sync with >> > what a QA or end user would use. Ie, you make 20 iterations of a >> > package to test something, tweak configs to check something, and get >> > it all working, but perhaps your machine is no longer setup the way a >> > fresh install or upgrade of your package would be. Or you tested a >> > version and then changed just 'one little thing' and pushed that and >> > it turned out to break it. >> >> Both hughsie and myself, and I think everyone else in favour of >> maintainer +1s, suggested maintainers should test *in a vanilla >> environment*, with the actual package they were submitting, before >> +1ing. +1ing on the basis of a test build or in a dirty environment >> would be a no-no and could lead to the removal of +1 privileges if >> repeated. > > I think we should define what a "vanilla environment" is then. One could > argue that either of the following could be described as "vanilla": > > - A fresh system without any modifications or unstable updates other > than that being tested. Pro: makes testing comparable. Contra: You > essentially need a special VM for testing which needs to be installed > freshly for each tested update. Makes tests comparable ;-), i.e. reduces > the amount of different environments in which an update is tested. I do > actually want testing to be done on systems that aren't just "minimal > install plus updates plus a user beside root". > > - A system in good condition (packages verify well, no dupes) that's > used normally, i.e. what you would see being used by normal persons > without any fancy hacks in configuration, or worse, non-config files > owned by packages. Pro: Easy to test as you don't need to do anything > fancy, just yum --enablerepo=updates-testing update <pkg>; <use pkg> Neither one of those definitions addresses the large variety of configurations that are possible with vanilla Fedora packages. E.g. your update might work wonderfully running a default Gnome desktop install, but crash portions of the KDE or XFCE stack (for cases of underlying desktop infrastructure). I don't think a maintainer can realistically replace wide-spread user based testing in a variety of environments. In light of that, we can either accept a maintainer +1 as "I tested this as I would use it and it worked" (which should be implied by them submitting the update already anyway), or we can disallow it as the policy says. I don't think adding more definitions or steps to the existing policy is really going to improve anything. josh -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
