----- "David Huff" <dhuff@xxxxxxxxxx> wrote:
> Michael Goldish wrote:
> > ----- "Lucas Meneghel Rodrigues" <lmr@xxxxxxxxxx> wrote:
> >
> >> Currently we have our kvm test control file and configuration
> file,
> >> having them split like this makes it harder for users to edit it,
> >> let's
> >> say, using the web frontend.
> >>
> >> So it might be good to merge the control file and the config file,
> >> and
> >> make a refactor on the control file code. Do you think this would
> be
> >> a valid approach? Any comments are welcome.
> >>
> >> Lucas
> >
> > What exactly do you mean by merge? Embed the entire config file in
> > the control file as a python string?
> >
> > A few comments:
> >
> > 1. The bulk of the config file usually doesn't need to be modified
> > from the web frontend, IMO. It actually doesn't need to be modified
> > very often -- once everything is defined, only minor changes are
> > required.
> >
> > 2. Changes to the config can be made in the control file rather
> easily
> > using kvm_config methods that are implemented but not currently
> used.
> > Instead of the short form:
> >
> > list = kvm_config.config(filename).get_list()
> >
> > we can use:
> >
> > cfg = kvm_config.config(filename)
> >
> > # parse any one-liner like this:
> > cfg.parse_string("only nightly")
> >
> > # parse anything the parser understands like this:
> > cfg.parse_string("""
> > install:
> > steps = blah
> > foo = bar
> > only qcow2.*Windows
> > """)
> >
> > # we can parse several times and the effect is cumulative
> > cfg.parse_string("""
> > variants:
> > - foo:
> > only scsi
> > - bar:
> > only WinVista.32
> > variants:
> > - 1:
> > - 2:
> > """)
> >
> > # we can also parse additional files:
> > cfg.parse_file("windows_cdkeys.cfg")
> >
> > # finally, get the resulting list
> > list = cfg.get_list()
> >
> > 3. We may want to consider something in between having the control
> and
> > config completely separated (what we have today), and having them
> both
> > in the same file. For example, we can define the test sets
> (nightly,
> > weekly, fc8_quick, custom) in the config file, and select the test
> set
> > (e.g. "only nightly") in the control file by convention.
> Alternatively
> > we can omit the test sets from the config file, and just define a
> single
> > test set (the one we'll be using) in the control file, or define
> several
> > test sets in the control file, and select one of them.
> > We can actually do both things at the same time, by defining the
> test
> > sets in the config file, and defining a "full" test set among them
> (I
> > think it's already there), which doesn't modify anything. If we want
> to
> > use a standard test set from the config file, we can do "only
> nightly"
> > in the control, and if we want to use a custom test set, we can do:
> > cfg.parse_string("""
> > only full
> > # define the test set below (no need for variants)
> > only RHEL
> > only qcow2
> > only autotest.dbench
> > """)
> >
> > 4. It could be a good idea to make a "windows_cdkeys.cfg" file,
> that
> > contains mainly single-line exceptions, such as:
> > WinXP.32: cdkey = REPLACE_ME
> > WinXP.64: cdkey = REPLACE_ME
> > Win2003.32: cdkey = REPLACE_ME
> > ...
> > The real cdkeys should be entered by the user. Then the file will
> be
> > parsed after kvm_tests.cfg, using the parse_file() method (in the
> > control). This way the user won't have to enter the cdkeys into the
> > long config file every time it gets replaced by a newer version.
> The
> > cdkeys file won't be replaced because it's specific to the test
> > environment (we'll only supply a sample like we do with
> kvm_tests.cfg).
> >
> > Maybe we can generalize this idea and call the file
> local_prefs.cfg,
> > and decide that the file should contain any environment-specific
> > changes that the user wants to make to the config. The file will
> > contain mainly exceptions (single or multi-line). But I'm not sure
> > there are many environment specific things other than cdkeys, so
> maybe
> > this isn't necessary.
> >
> >
> > Let me know what you think.
>
> Michael all very good comments, I specifically like the windows
> config
> file idea.
>
> The way I always envisioned it was something like this......
>
> The config file specifies the whole test matrix, ie all variants that
> you could run each test on, ie all os's, all archs, all disk types,
> all
> cpu/mem configurations.
>
> The control file would be more of a test specific config file,
> setting
> any local or environmental vars for each test, and like Michael said
> can
> override "stuff" fromt he main confg file...
>
> I also really like the idea of creating a generic kvm_test that all
> kvm
> tests would inherent from, ie.
> $AUTOTEST/client/common_lib/kvm_test.py
>
> All helper classes ie. kvm.py, kvm_utils.py, kvm_config.py, and even
> the
> config file itself could then go into
> $AUTOTEST/client/common_lib/test_utils/ or even maybe something like
> $AUTOTEST/client/common_lib/kvm_test_utils/
>
> All kvm specific tests would inherent form the generic kvm_test, and
> then go into either $AUTOTEST/client/tests/ or
> $AUTOTEST/client/kvm_tests/ directorys, each having their own sub dir
> like the current autotest tests. In this dir there would be a
> control
> file specific for each test, that can override the full test matrix
> descried inthe the generic kvm_tests.cfg, as well as any additional
> file
> required by the test.
>
> Anyway just some of my thoughts, I know its great in theory however
> may
> have some implementation short falls, like interdependence between
> tests
> and such...
>
>
> Comments..
I think I understand your suggestion, but let me make sure:
- If there's a global config file that is shared by all tests, I suppose
it'll run all the tests one by one, right?
- Where will test sets be defined -- in the global config file?
- If each individual test inherits from the global config file, it'll
also inherit dictionaries describing other tests, right?
e.g. the configuration of the boot test must explicitly state "only boot",
or it'll run install, migration and autotest as well?
- If you run the control file of a specific test, what happens -- does
that specific test run in many configurations (many guests, cpu options,
network, ide/scsi), or does it run just once with a single configuration?
I suppose the "normal" behavior would be to run in many configurations, but
I'm not sure what your intention was.
- Will the global config file look like the config files we have today?
I think this should be possible to implement, but I haven't given it much
thought so I'm not sure. The more interesting question is whether it's a
good idea. What are the advantages over the current approach?
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
