On Fri, Apr 08, 2016 at 02:47:19PM -0700, Adam Williamson wrote: > On Fri, 2016-04-08 at 12:03 -0700, Brian C. Lane wrote: > > On Thu, Apr 07, 2016 at 03:33:24PM -0700, Adam Williamson wrote: > > > > > > ## What could we possibly do with this? > > > > > > There's a few options. > > > > > > 1) We could keep the jenkins+official runner setup that anaconda's > > > using for doing testing at the level of the anaconda project, and just > > > use this to run the kickstart tests at the Fedora QA distribution > > > testing level. So that whenever we (QA) run the openQA tests for > > > whatever reason, the kickstart tests get run, but keep it as a QA > > > thing. > > > > > > This is kinda the easiest to do short term (especially the easiest for > > > anaconda folks as you pretty much have to do nothing), but medium and > > > long term, it might be a bit more error prone than the other > > > approaches. I suspect the way it would work out, anaconda team would > > > really only care about keeping the tests running in the 'official' > > > runner, and we (QA) would have to spend quite a bit of time keeping the > > > converter in line with the official runner's behaviour. > > > > > > We'd also probably have to get a bit more hardware from somewhere for > > > this case, because suddenly adding another ~50 tests to the set of > > > tests run in the current QA instance will make its test runs quite a > > > lot longer; if we get another box or two we can add more workers to > > > compensate for this. > > > > > > 2) We could try to have QA act as a 'service provider' for anaconda to > > > run the tests in openQA: basically in this scenario we'd give you the > > > necessary credentials to trigger openQA tests in an openQA instance > > > that's owned/maintained by QA. There would be a lot of details to > > > figure out in this case. > > > > > > 3) We could set anaconda up with its own openQA instance for running > > > the tests, and we'd just share tooling stuff like the ansible plays and > > > the openQA scheduling code (however much turns out to be in common). > > > This would mean someone in anaconda team being basically familiar with > > > care and feeding of an openQA instance, which isn't that hard. You'd > > > also need the hardware to run it on. > > > > > > In scenarios 2) and 3), I guess we'd burn down the current kickstart- > > > tests runner, and we could start tweaking the way the tests are > > > implemented (in terms of how prep and asset creation are specified and > > > actually carried out) to fit in more nicely with openQA's design. In > > > scenario 1), kickstart-tests would be pretty much unmodified and we'd > > > have to keep working on the converter to keep its behaviour in line > > > with the official stuff. We *could* invent some kind of intermediate > > > format for specifying test requirements which would be interpreted > > > differently by the openQA and 'native' runners, but I suspect that > > > would sound like a lot of engineering. > > > If you can automate all the things I'd say it's useful to have openQA > > run these. > > > > I don't want to see the current system go away though, it is pretty > > useful to be able to run the tests locally before pushing big changes > > out to the world. > > So to understand the local workflow: you have to build an image with > all your changes and run the tests on that, right? > > It would be possible to do this with openQA, even without having your > own pet openQA instance - though note it's also quite easy to set up a > pet openQA instance, you don't have to do a full-fat bare metal > deployment like openqa.happyassassin.net , we have a documented > workflow for doing it in docker containers: > > https://bitbucket.org/rajcze/openqa_fedora_tools/src/a8c83b6929bb4935697b34ae36f56e0d0dae96ce/docker/?at=develop > > this is how jskladan and garretraziel work, so it's a pretty safe bet > to keep working. Looks interesting, I'll keep that in mind. > > Regardless, so long as you have an image file and can get it to the > openQA server somehow (or put it up on a web server), you can quite > trivially tell openQA 'hey, run these tests on this image'. If we Right, it's just easier/faster for me to make my boot.iso here and run the tests on it than it is to make it and wait to upload it someplace. > wanted to let people run scratch tests on a shared openQA instance we > might have to think about ways to do naming and stuff such that there's > a clear separation between those and the 'production' tests, or maybe > have separate instances for scratch purposes, but all that is solvable. > (Depending on the level of activity, it may require someone somewhere > to cough up some money, though :>) > > I also set up a thing in our openQA scheduling code last week for > letting you run tests with an arbitrary updates.img , which you could > use for quick and dirty tests too. That would be pretty useful, usually that's all it needs, not a full rebuild of the iso. -- Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT) _______________________________________________ Anaconda-devel-list mailing list Anaconda-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/anaconda-devel-list
