On Tue, Apr 05, 2011 at 08:07:03AM -0700, Chris Wright wrote: [snip] > kvm-autotest > - roadmap...refactor to centralize testing (handle the xen-autotest split off) > - internally at RH, lmr and cleber maintain autotest server to test > branches (testing qemu.git daily) > - have good automation for installs and testing > - seems more QA focused than developers > - plenty of benefit for developers, so lack of developer use partly > cultural/visibility... > - kvm-autotest team always looking for feedback to improve for > developer use case > - kvm-autotest day to have folks use it, write test, give feedback? > - startup cost is/was steep, the day might be too much handholding > - install-fest? (to get it installed and up and running) > - buildbot or autotest for testing patches to verify building and working > - one goal is to reduce mailing list load (patch resubmission because > they haven't handled basic cases that buildbot or autotest would have > caught) > - fedora-virt test day coming up on April 14th. lucas will be on hand and > we can piggy back on that to include kvm-autotest install and virt testing > - kvm autotest run before qemu pull request and post merge to track > regressions, more frequent testing helps developers see breakage > quickly > - qemu.git daily testing already, only the "sanity" test subset > - run more comprehensive "stable" set of tests on weekends > - one issue is the large number of known failures, need to make these > easier to identify (and fix the failures one way or another) > - create database and verify (regressions) against that > - red/yellow/green (yellow shows area was already broken) > - autotest can be run against server, not just on laptop > - how to do remote client display testing (e.g. spice client) > - dogtail and LDTP > - graphics could be tested w/ screenshot compares > - WHQL testing automated as well screenshots are already there, and they are a great start. But you can't really do testing if you aren't recreating the same environment, and having a client server where there is no client, while being a good test, doesn't cover the case where the client is connected :) So I was basically talking about the added requirement of creating a client connection (one or more) to a single vm. Note that I wasn't asking anyone to develop this - I'm just asking if patches in that direction would be accepted/interesting. We (well, Swapna, cc-ed) are still working on deciding exactly how to do automated testing as a project. Regarding the dogtail/LDTP issue, they are about specific tests run inside the guest, and they are certainly something we would leverage. But like I mentioned on the call, there is a whole suite of whql tests that are display specific, and don't require anything new. In fact, a few months ago I added support for autotest to run one of them, resizing to all the possible modes - so I know I don't need dogtail for significant portions of our testing. (sorry, no git link - I'll clean it up and post, it's been done 10 months ago so probably won't cleanly apply :) For some ideas about what we are interested in see http://spice-space.org/page/SpiceAutomatedTesting. (just the Requirements section). > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
