Hi all,
We've recently implemented a very simple form of parallel test execution into KVM-Autotest and we'd like some feedback on it. This suggestion allows the user to manually assign tests to hosts/queues. It also takes care of assigning different MAC address ranges to hosts/queues. By 'queues' I mean parallel execution pipelines. Each host has one or more queues. The number of queues is defined by the user, and should reflect the capabilities of the host.
This implementation involves only minor modifications to the code itself; most of the work is done in a new config file kvm_hosts.cfg, which has the exact same format as kvm_tests.cfg. The new file provides the framework with information about hosts/queues. The new file is parsed after kvm_tests.cfg. The test sets (such as 'nightly' and 'weekly'), previously defined at the end of kvm_tests.cfg, should now be defined last, after kvm_hosts.cfg. Test sets no longer select only the tests to execute, but also where each test should be executed (i.e. on what host/queue).
The final result of parsing the config files is a list of tests, each with its own 'hostname' and 'queue' parameters. Each host executes only the tests whose 'hostname' parameter matches the current host, and puts tests with different 'queue' values in parallel pipelines of execution.
Ideally, the Autotest server should take care of assigning tests to hosts automatically, but there are still a few technical difficulties to be resolved before we can implement that. We're considering the current suggestion as a temporary solution until a better one is found.
Basically, the advantages are:
- allows the user full control over what tests run, and where/how they run
- takes care of assigning MAC address ranges to different hosts/queues (required for TAP networking)
- can be used from the server or with the client, which makes it relevant also for users who don't have an Autotest server installed
- involves only minor code changes (except for the config files)
- is pretty much the simplest possible solution (and simple is good)
Drawbacks:
- requires some initial work to be done by the user -- the user has to define exactly where each test should run
- test sets need to be modified when tests or hosts are added/removed, to include/exclude them
We'd like to get some feedback on this -- is it usable, comfortable, do you have any suggestions, etc. We'll also be happy to answer questions.
There are no actual patches attached to this message because the implementation is trivial and we want to focus on the user's point of view.
The following is a sample kvm_hosts.cfg file.
Note that kvm_hosts.cfg can be structured in many different ways that will achieve the same results.
# Start by parsing the regular config file
include kvm_tests.cfg
# Now define MAC address ranges for VMs
# (this is environment specific information)
base_mac_r0 = 00:00:00:00:00:00
base_ip_r0 = 10.20.0.0
range_size_r0 = 256
base_mac_r1 = 00:00:00:00:01:00
base_ip_r1 = 10.20.1.0
range_size_r1 = 256
base_mac_r2 = 00:00:00:00:02:00
base_ip_r2 = 10.20.2.0
range_size_r2 = 256
# (define more MAC address ranges here...)
# Define 4 parallel queues (or more)
variants:
- @queue1:
queue = 1
# Each queue uses a different environment file
env = env
- @queue2:
queue = 2
env = env2
- @queue3:
queue = 3
env = env3
- @queue4:
queue = 4
env = env4
# Define the hosts to be used
# (this is where you provide host-specific information)
variants:
- @hostname1:
# 'hostname1' should be a real hostname
hostname = hostname1
# Let hostname1 run 2 parallel test queues
only queue1 queue2
queue1:
# Assign range r0 to queue1 on hostname1
mac_ranges = r0
queue2:
# Assign range r1 to queue2 on hostname1
mac_ranges = r1
- @hostname2:
# Let hostname2 run a single test queue
hostname = hostname2
only queue1
# Assign range r2 to hostname2
mac_ranges = r2
# (define more hosts here...)
The following is a sample test set definition:
variants:
# This is the 'nightly' test set
- @nightly:
# Select the hosts to run tests on
only hostname1 hostname2
# Select the tests to run
only qcow2
only ide
only RHEL|Windows
# Assign tests to individual hosts
hostname1:
# hostname1 will run Windows tests
only Windows
# Select the tests to run on each queue
# (the selections should be mutually-exclusive)
queue1:
only Win2000 WinXP Win2003
queue2:
only WinVista Win2008
hostname2:
# hostname2 has a single queue,
# and it will run RHEL tests
only RHEL
# This is the 'weekly' test set
- @weekly:
...
Finally, one should select the desired test set (e.g. 'only nightly') somewhere -- at the end of a config file or in the control file (using config.parse_string()).
Thanks,
Michael
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
