Re: Running kickstart-tests in openQA: should we?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Apr 07, 2016 at 03:33:24PM -0700, Adam Williamson wrote:
> ## What could we possibly do with this?
> 
> There's a few options.
> 
> 1) We could keep the jenkins+official runner setup that anaconda's
> using for doing testing at the level of the anaconda project, and just
> use this to run the kickstart tests at the Fedora QA distribution
> testing level. So that whenever we (QA) run the openQA tests for
> whatever reason, the kickstart tests get run, but keep it as a QA
> thing.
> 
> This is kinda the easiest to do short term (especially the easiest for
> anaconda folks as you pretty much have to do nothing), but medium and
> long term, it might be a bit more error prone than the other
> approaches. I suspect the way it would work out, anaconda team would
> really only care about keeping the tests running in the 'official'
> runner, and we (QA) would have to spend quite a bit of time keeping the
> converter in line with the official runner's behaviour.
> 
> We'd also probably have to get a bit more hardware from somewhere for
> this case, because suddenly adding another ~50 tests to the set of
> tests run in the current QA instance will make its test runs quite a
> lot longer; if we get another box or two we can add more workers to
> compensate for this.
> 
> 2) We could try to have QA act as a 'service provider' for anaconda to
> run the tests in openQA: basically in this scenario we'd give you the
> necessary credentials to trigger openQA tests in an openQA instance
> that's owned/maintained by QA. There would be a lot of details to
> figure out in this case.
> 
> 3) We could set anaconda up with its own openQA instance for running
> the tests, and we'd just share tooling stuff like the ansible plays and
> the openQA scheduling code (however much turns out to be in common).
> This would mean someone in anaconda team being basically familiar with
> care and feeding of an openQA instance, which isn't that hard. You'd
> also need the hardware to run it on.
> 
> In scenarios 2) and 3), I guess we'd burn down the current kickstart-
> tests runner, and we could start tweaking the way the tests are
> implemented (in terms of how prep and asset creation are specified and
> actually carried out) to fit in more nicely with openQA's design. In
> scenario 1), kickstart-tests would be pretty much unmodified and we'd
> have to keep working on the converter to keep its behaviour in line
> with the official stuff. We *could* invent some kind of intermediate
> format for specifying test requirements which would be interpreted
> differently by the openQA and 'native' runners, but I suspect that
> would sound like a lot of engineering.

If you can automate all the things I'd say it's useful to have openQA
run these.

I don't want to see the current system go away though, it is pretty
useful to be able to run the tests locally before pushing big changes
out to the world.

-- 
Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT)

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/anaconda-devel-list



[Index of Archives]     [Kickstart]     [Fedora Users]     [Fedora Legacy List]     [Fedora Maintainers]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [Yosemite Photos]     [KDE Users]     [Fedora Tools]
  Powered by Linux