On Mon, 2015-11-02 at 10:52 -0700, Kevin Fenzi wrote: > > > Perhaps full release validation could occur on composes every two > > weeks for branched releases? Or is that still too demanding? > > No idea. I don't want to speak for the QA folks. So to put it simply and broadly: there's a trade-off between how often we want to release, and how heavily we can test those releases. Frankly, no, I don't think it's viable to do the entire current five- ring circus release validation process every two weeks. It's not just a question of QA doing the testing, because of course testing without fixes is meaningless: it's a question of QA, releng, and maintainers all being perpetually in the 'ah crap it's two weeks before release' frame of mind. Which isn't really sustainable on the current level, I don't think. But then, that doesn't mean we can't change things, it just means we re-calibrate our expectations. We can release more often if we want to, sure: we just have to be okay with each of those releases getting somewhat less testing. As Kevin noted upthread there *are* various ways in which we're tackling automation of testing, which does help here. You can sketch a beautiful picture where we build the distribution on CI principles: we run a whole bunch of automated tests any time anyone sneezes on a package, and anything that breaks gets rejected, and we can thus spin a new release from the 'approved' bits any damn time we please and be sure that it'd work. But, also as Kevin noted, I don't think we can ever really achieve that perfectly; there's just too much in distribution testing which isn't fully susceptible to automation, I think at least in practical terms in the medium-term future there's always gonna need to be some humans in the loop to some degree. That doesn't mean we can't try and move in that direction, though. We can keep building out the automation efforts to the point where we can be pushing out 'releases' that we're fairly sure are at least basically working very frequently. They wouldn't be as heavily tested as our current traditional heavy 'releases' are, but they also wouldn't be unknown quantities. In fact, since Flock, we've more or less been doing this - any time you see a 'compose check report' email with a pretty small number of openQA failures, you can grab that nightly build and be reasonably sure that it will at least install and let you log in, hardware issues excepted (openQA is all virt testing). There are short- term plans already in place to improve this general area continuously: releng is moving towards the nightly composes being much more like TCs and RCs, QA is working to move openQA public, give it more resources, and add more tests, Taskotron is definitely coming to a point soon where it'll be feasible to do more forms of release testing in it, and there are other mechanisms we're looking at too. So yeah, if we want to start in *some* form releasing stuff more frequently, it's possible. We have the technology, and we're making it better quite frequently. I believe there are plans this cycle to regularly publish updated Cloud (including Atomic host) images, which will certainly form a starting point. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
