On Tue, Jul 21, 2009 at 11:37 AM, David Huff<dhuff@xxxxxxxxxx> wrote: > Michael all very good comments, I specifically like the windows config > file idea. > > The way I always envisioned it was something like this...... > > The config file specifies the whole test matrix, ie all variants that > you could run each test on, ie all os's, all archs, all disk types, all > cpu/mem configurations. > > The control file would be more of a test specific config file, setting > any local or environmental vars for each test, and like Michael said can > override "stuff" fromt he main confg file... It's not to say that separating the test matrix definition from the test job logic is a bad idea organization-wise, however like I pointed out on Michael's and Ryan's comments, it sort of defeats the original autotest design goals (that was pointed out by Martin later on this thread, I'll get to it later). > I also really like the idea of creating a generic kvm_test that all kvm > tests would inherent from, ie. $AUTOTEST/client/common_lib/kvm_test.py > > All helper classes ie. kvm.py, kvm_utils.py, kvm_config.py, and even the > config file itself could then go into > $AUTOTEST/client/common_lib/test_utils/ or even maybe something like > $AUTOTEST/client/common_lib/kvm_test_utils/ Yes, moving the large amount of infrastructure developed for the kvm test into the autotest library namespace is something I also want to do at some point in time. Good point. > All kvm specific tests would inherent form the generic kvm_test, and > then go into either $AUTOTEST/client/tests/ or > $AUTOTEST/client/kvm_tests/ directorys, each having their own sub dir > like the current autotest tests. In this dir there would be a control > file specific for each test, that can override the full test matrix > descried inthe the generic kvm_tests.cfg, as well as any additional file > required by the test. Yes, moving tests out of the kvm_test realm is good, since it keeps things more manageable on the kvm test itself. However, we need to take into account non-linux guests, where we can't just yet use raw autotest. > Anyway just some of my thoughts, I know its great in theory however may > have some implementation short falls, like interdependence between tests > and such... Yes, at some point, I want to take more advantage of server based testing. We would have a server control file that: 1) A test that sets up a test client (A physical box running linux) to be a virtualization host (KvmHost) and creates guests. The result of this test must be a number of KvmGuests, that can be later used to perform tests on it. 2) Just instantiate autotest inside the KvmGuests and run tests on them using the server side control API. Easier said than done, because we will deal with windows guests, test dependency and controlling hypervisor parameters. So I am not saying that this is something we need to jump in right now. My main concerns right now are: 1) Make sure the infrastructure we've got now works really well 2) Grow the amount of tests we have so we can increase KVM testing coverage Working out architectural changes would come later, at a more appropriate time. Refactoring the control file and the test config file is one of the improvements we can make on the short term using the client model, so that's why I started the discussion. Thanks for your thoughts on that David. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
