On Sat, Nov 20, 2010 at 03:35:43PM -0700, Kevin Fenzi wrote: > ok, I dug through the devel list for the last month or two and wrote > down all the various ideas folks have come up with to change/improve > things. > > Here (in no particular order) are the ideas and some notes from me on > how we could enable them. Please feel free to add new (actual/concrete > ideas or notes): > > * Just drop all the requirements/go back to before we had any updates > criteria. > > * Change FN-1 to just security and major bugfix > > This may be hard to enforce or figure out if something is a major bugfix. > > * allow packages with a %check section to go direct to stable > > Bodhi would have to have a list or some way to note these packages, > it would also need to change as they were added/removed. Perhaps it could > just be an AutoQA +1 for having a check section? > On the con side, some checks may be simple and may not note things > that are fedora only issues. > > * setup a remote test env that people could use to test things. > > This would be lovely with some kind of cloud setup. Have a set of base kickstart > files and a package/tester could request an instance, install the update, test, and then > the instance would be removed. We would need a timelimit of some kind, > and not sure there is any HW in infrastructure for this, but it would sure be cool. ;) > Perhaps working with the cloud sig we could use EC2 instances for this? > > * require testing only for packages where people have signed up to be testers > > Packages without 'official' testers could bypass testing or have some lower karma > requirement. We would need for this a list of packages that have had people sign > up to test. > > * Ask maintainers to provide test cases / test cases in wiki for each package? > > Test cases are not easy to make, many maintainers won't can't do so, but > it would be lovely to have even a base checklist of things that should work > in the package everytime. > > * have a way to get interested testers notified on bodhi updates for packages > they care about. > > We would need to add some kind of tester list to pkgdb, and bodhi would need to be > able to get this to mail them when a update changed state. > We may not get many people signing up for some packages, but this might be a > good way to know what packages we have testers for and get them more involved > in testing. Ideally it could mail them on update submission at least. > > * reduced karma requirement on other releases when one has gone stable > > Bodhi would need to note when a update went to stable if the exact same > version (with dist tag differences) was in testing for other releases. > It could then allow less karma to go stable, or add +1 from the other > update going stable. > > Other concrete ideas? > Security update ideas: * Security updates may push directly to stable -- this would depend on our weighing of costs: a security update may be broken. OTOH, if we don't get security updates out fast, is that a worse danger to our users? I think I'd like to try one of the other options below first but those require a bit more work/coordination so we need to think of the cost to implement as well. * Ask the QA team to commit to testing security updates. The idea is that updates flagged security have a dedicated pool of testers that will check them and get them out ASAP. * Security updates have a small timeout period -- there's two questions here: Is a week (as is currently the case for non-critpath) too long? Can wehave a timeout even for critpath packages so a security update doesn't get held up forever? Critpath ideas: * critpath package timeout -- let's have a timeout period even for critpath packages. Perhaps this could be longer than for non-critpath. The big issue that people have observed with depending on timeouts, though, is that pushing new updates in the meantime resets the time that a package needs to wait to get to stable. Could bodhi be changed to let multiple packages be in the testing repository at one time and only obsolete them when a newer package enters the stable repo? That would allow longer timeout periods to make more sense. Lack of manpower ideas: * Allow anonymous karma to count. Anonymous karma would allow more people who report bugs in bugzilla to add karma in bodhi without having to get a second account in the Fedora Account System. For critpath packages, we're already mandating that a proventester must give karma so we're making sure that someone with an account is giving feedback there. * Do we believe that there's value in silent testing (knowing that people have a package installed from testing but have not submitted karma either way?) If so, create a tool that ties into yumdb and if the user opts in, submits that data to us. Then add karma based upon that data. Variation on an option you recorded: * Drop testing requirement until AutoQA can give karma. Rework based on what tests auto qacan perform. -Toshio
Attachment:
pgp7i1HyNPOrV.pgp
Description: PGP signature
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
