On Fri, Apr 08, 2011 at 09:58:22AM -0300, Lucas Meneghel Rodrigues wrote: > On Thu, 2011-04-07 at 11:03 +0100, Stefan Hajnoczi wrote: > > On Tue, Apr 5, 2011 at 6:37 PM, Lucas Meneghel Rodrigues <lmr@xxxxxxxxxx> wrote: > > >> Perhaps kvm-autotest is a good platform for the automated testing of > > >> ARM TCG. Paul is CCed, I recently saw the Jenkins qemu build and boot > > >> tests he has set up. Lucas, do you have ideas on how these efforts > > >> can work together to bring testing to upstream QEMU? > > >> http://validation.linaro.org/jenkins/job/qemu-boot-images/ > > > > > > I heard about jenkins before and it is indeed a nice project. What they > > > do here, from what I could assess browsing at the webpage you provided > > > is: > > > > > > 1) Build qemu.git every time there are commits > > > 2) Boot pre-made 'pristine' images, one is a lenny arm image and the > > > other is a linaro arm image. > > > > > > It is possible to do the same with kvm autotest, just a matter of not > > > performing guest install tests and executing only the boot tests with > > > pre-made images. What jenkins does here is a even quicker and shorter > > > version of our sanity jobs. > > > > > > About how we can work together, I thought about some possibilities: > > > > > > 1) Modify the jenkins test step to execute a kvm autotest job after the > > > build, with the stripped down test set. We might gain some extra debug > > > info, that the current test step does not seem to provide > > > 2) Do the normal test step and if that succeeds, trigger a kvm autotest > > > job that does more comprehensive testing, such as migration, time drift, > > > block layer, etc > > > > > > The funny thing is that KVM autotest has infrastructure to do the same > > > as jenkins does, but jenkins is highly streamlined for the buildbot use > > > case (continuous build and integration), and I see that as a very nice > > > advantage. So I'd rather keep use jenkins and have kvm autotest plugged > > > into it conveniently. > > > > That sounds good. I think the benefit of working together is that > > different entities (Linaro, Red Hat, etc) can contribute QEMU tests > > into a single place. That testing can then cover to both upstream and > > downstream to prevent breakage. > > > > So kvm-autotest can run in single job mode and kicked off from jenkins > > or buildbot? > > > > It sounds like kvm-autotest has or needs its own cron, result > > archiving, etc infrastructure. Does it make sense to use a harness > > like jenkins or buildbot instead and focus kvm-autotest purely as a > > testing framework? > > In the context that there are already jenkins/buildbot servers running > for qemu, having only the KVM testing part (autotest client + kvm test) > is a possibility, to make things easier to plug and work with what is > already deployed. > > However, not possible to focus KVM autotest as a 'test framework'. What > we call KVM autotest is in actuality, a client test of autotest. > Autotest is a generic, large collection of programs and libraries > targeted at peforming automated testing on the linux platform, it was > developed to test the linux kernel itself, and it is used to do > precisely that. Look at test.kernel.org. All those tests are executed by > autotest. > > So, autotest is much more than KVM testing, and I am one of the autotest > maintainers, so I am commited to work on all parts of that stack. > Several testing projects urelated to KVM use our code, and our > harnessing and infrastructure is already pretty good, we'll keep to > develop it. > > The whole thing was designed in a modular way, so it's doable to use > parts of it (such as the autotest client and the KVM test) and integrate > with stuff such as jenkins and buildbot, and if people need and want to > do that, awesome. But we are going to continue develop autotest as a > whole framework/automation utilities/API, while developing the KVM test. I wasn't aware of the scope of autotest and your involvement. I need to look into autotest more :). Stefan -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
