* Mike Burns <mburns@xxxxxxxxxx> [2009-05-20 16:13]: > Ryan Harper wrote: > > * Michael Goldish <mgoldish@xxxxxxxxxx> [2009-05-17 09:50]: > > > >> Hi all, > >> > >> We've recently implemented a very simple form of parallel test > >> execution into KVM-Autotest and we'd like some feedback on it. This > >> suggestion allows the user to manually assign tests to hosts/queues. > >> It also takes care of assigning different MAC address ranges to > >> hosts/queues. By 'queues' I mean parallel execution pipelines. Each > >> host has one or more queues. The number of queues is defined by the > >> user, and should reflect the capabilities of the host. > >> > >> This implementation involves only minor modifications to the code > >> itself; most of the work is done in a new config file kvm_hosts.cfg, > >> which has the exact same format as kvm_tests.cfg. The new file > >> provides the framework with information about hosts/queues. The new > >> file is parsed after kvm_tests.cfg. The test sets (such as > >> 'nightly' and 'weekly'), previously defined at the end of > >> kvm_tests.cfg, should now be defined last, after kvm_hosts.cfg. > >> Test sets no longer select only the tests to execute, but also > >> where each test should be executed (i.e. on what host/queue). > >> > >> The final result of parsing the config files is a list of tests, each > >> with its own 'hostname' and 'queue' parameters. Each host executes > >> only the tests whose 'hostname' parameter matches the current host, > >> and puts tests with different 'queue' values in parallel pipelines > >> of execution. > >> > >> Ideally, the Autotest server should take care of assigning tests to > >> hosts automatically, but there are still a few technical difficulties > >> to be resolved before we can implement that. We're considering the > >> current suggestion as a temporary solution until a better one is > >> found. > >> > >> Basically, the advantages are: > >> - allows the user full control over what tests run, and where/how they run > >> - takes care of assigning MAC address ranges to different hosts/queues (required for TAP networking) > >> - can be used from the server or with the client, which makes it relevant also for users who don't have an Autotest server installed > >> - involves only minor code changes (except for the config files) > >> - is pretty much the simplest possible solution (and simple is good) > >> > >> Drawbacks: > >> - requires some initial work to be done by the user -- the user has to define exactly where each test should run > >> - test sets need to be modified when tests or hosts are added/removed, to include/exclude them > >> > > > > I took a slightly different approach. The kvm_tests.cfg file already > > provides a dependency relationship between different tests. I modified > > the main loop in the control file to walk the entire list of jobs and > > pull out any jobs that don't have any dependencies (ie, install tests). > > And then run N jobs in parallel from that list until it is exhausted; > > then store the results. Then loop the over the remaining list of jobs > > again finding the jobs that can be run. > > > > On a larger multi core system, one might set the number of parallel jobs > > equal to the number of cores. > > > > I think this works well with using autoserv to farm out different > > kvm_tests.cfg to different machines. > > > > Attaching my stale patch just for comment. Needs to be updated since I > > sat on this for a while. There were a number of issues: > > > > - kvm_log is a shared resource, fixed it up so parallel jobs can both > > call it > > - vnc, redir and other network resources are shared, so, in > > kvm_tests.cfg file each job needs a parallel offset. > > - in kvm_tests.cfg file need to define additional vm and nic objects, > > one for each parallel threads. > > > > Advantages: > > - works a lot like the single threaded model does, and if threads=1 > > then it runs the same path > > - config files don't change significantly, just some additional > > VM objects at the top and some offset values > > - transparent to an autoserv setup, autoserv would just need to > > specify the kvm_tests.cfg file to each host. > > > > Disadvantages: > > - the main loop waits for each group of parallel jobs to complete > > before starting any more. If somehow an install is mixed with a > > reboot test, we'll wait around before starting more jobs > > - probably a few more here, but I don't have them on the top of my > > head. > > > > > I haven't looked through the code at all yet, but the design seems more > in line with my thoughts of parallel execution. Just a couple really > quick thoughts on this: > > There is an effort being made to merge kvm-autotest with autotest. I > think part of that includes a change to remove kvm_log. I don't know if > the upstream logging system will support multi-threading, but I wouldn't > spend any time working on kvm_log. Indeed, I spent as little time on it as it took to get it working with 2 parallel installs. I don't have any plans for it other than if it is present to not be in the way of parallel execution. > > I haven't dug through all the execution code yet. I've mostly played > around in the kvm_install area and stepsfile stuff. Couldn't we do > something like this: > > define a param called num_threads > have base names for things like vms, images, nics, etc like vm, image, nic > each thread picks up a test to run and uses the base names + thead num > We could then use the thread number all the offsets as well. Yep. -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh@xxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
