On Fri, 2016-04-08 at 12:03 -0700, Brian C. Lane wrote: > On Thu, Apr 07, 2016 at 03:33:24PM -0700, Adam Williamson wrote: > > > > ## What could we possibly do with this? > > > > There's a few options. > > > > 1) We could keep the jenkins+official runner setup that anaconda's > > using for doing testing at the level of the anaconda project, and just > > use this to run the kickstart tests at the Fedora QA distribution > > testing level. So that whenever we (QA) run the openQA tests for > > whatever reason, the kickstart tests get run, but keep it as a QA > > thing. > > > > This is kinda the easiest to do short term (especially the easiest for > > anaconda folks as you pretty much have to do nothing), but medium and > > long term, it might be a bit more error prone than the other > > approaches. I suspect the way it would work out, anaconda team would > > really only care about keeping the tests running in the 'official' > > runner, and we (QA) would have to spend quite a bit of time keeping the > > converter in line with the official runner's behaviour. > > > > We'd also probably have to get a bit more hardware from somewhere for > > this case, because suddenly adding another ~50 tests to the set of > > tests run in the current QA instance will make its test runs quite a > > lot longer; if we get another box or two we can add more workers to > > compensate for this. > > > > 2) We could try to have QA act as a 'service provider' for anaconda to > > run the tests in openQA: basically in this scenario we'd give you the > > necessary credentials to trigger openQA tests in an openQA instance > > that's owned/maintained by QA. There would be a lot of details to > > figure out in this case. > > > > 3) We could set anaconda up with its own openQA instance for running > > the tests, and we'd just share tooling stuff like the ansible plays and > > the openQA scheduling code (however much turns out to be in common). > > This would mean someone in anaconda team being basically familiar with > > care and feeding of an openQA instance, which isn't that hard. You'd > > also need the hardware to run it on. > > > > In scenarios 2) and 3), I guess we'd burn down the current kickstart- > > tests runner, and we could start tweaking the way the tests are > > implemented (in terms of how prep and asset creation are specified and > > actually carried out) to fit in more nicely with openQA's design. In > > scenario 1), kickstart-tests would be pretty much unmodified and we'd > > have to keep working on the converter to keep its behaviour in line > > with the official stuff. We *could* invent some kind of intermediate > > format for specifying test requirements which would be interpreted > > differently by the openQA and 'native' runners, but I suspect that > > would sound like a lot of engineering. > If you can automate all the things I'd say it's useful to have openQA > run these. > > I don't want to see the current system go away though, it is pretty > useful to be able to run the tests locally before pushing big changes > out to the world. So to understand the local workflow: you have to build an image with all your changes and run the tests on that, right? It would be possible to do this with openQA, even without having your own pet openQA instance - though note it's also quite easy to set up a pet openQA instance, you don't have to do a full-fat bare metal deployment like openqa.happyassassin.net , we have a documented workflow for doing it in docker containers: https://bitbucket.org/rajcze/openqa_fedora_tools/src/a8c83b6929bb4935697b34ae36f56e0d0dae96ce/docker/?at=develop this is how jskladan and garretraziel work, so it's a pretty safe bet to keep working. Regardless, so long as you have an image file and can get it to the openQA server somehow (or put it up on a web server), you can quite trivially tell openQA 'hey, run these tests on this image'. If we wanted to let people run scratch tests on a shared openQA instance we might have to think about ways to do naming and stuff such that there's a clear separation between those and the 'production' tests, or maybe have separate instances for scratch purposes, but all that is solvable. (Depending on the level of activity, it may require someone somewhere to cough up some money, though :>) I also set up a thing in our openQA scheduling code last week for letting you run tests with an arbitrary updates.img , which you could use for quick and dirty tests too. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ Anaconda-devel-list mailing list Anaconda-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/anaconda-devel-list
