Stracing libvirtd shows that the qemu driver is executing 2 different
qemu binaries 3 times each to fetch the version, capabilities [supported
devices], and cpu models each time VM state is queried. E.g., [lines wrapped]:
6471 17:15:26.561890 execve("/usr/bin/qemu",
["/usr/bin/qemu", "-cpu", "?"], [/* 2 vars */]) = 0
6472 17:15:26.626668 execve("/usr/bin/qemu",
["/usr/bin/qemu", "-help"], [/* 2 vars */]) = 0
6473 17:15:26.698104 execve("/usr/bin/qemu",
["/usr/bin/qemu", "-device", "?", "-device", "pci-assign,?",
"-device", "virtio-blk-pci,?"], [/* 2 vars */]) = 0
6484 17:15:27.267770 execve("/usr/bin/qemu-system-x86_64",
["/usr/bin/qemu-system-x86_64", "-cpu", "?"],
/* 2 vars */]) = 0
6492 17:15:27.333177 execve("/usr/bin/qemu-system-x86_64",
["/usr/bin/qemu-system-x86_64", "-help"],
[/* 2 vars */]) = 0
6496 17:15:27.402280 execve("/usr/bin/qemu-system-x86_64",
["/usr/bin/qemu-system-x86_64", "-device", "?", "-device",
"pci-assign,?", "-device", "virtio-blk-pci,?"],
[/* 2 vars */]) = 0
~1sec per libvirt api call. Not a killer, but on a heavily loaded
host -- several 10s of VMs -- a periodic query of all VM state, such
as from a cloud compute manager, can take a couple of minutes to
complete.
Because the qemu binaries on the host do not change all that often,
the results of parsing the qemu help output from the exec's above
can be cached. The qemu driver already does some caching of
capabilities, but it does not prevent the execs above.
This series is a work in progress. I'm submitting it as an RFC because I
saw Eric mention the frequent execing of qemu binaries and I have been
working on this to eliminate the overhead shown above.
The series caches the parse results of:
+ qemuCapsExtractVersionInfo
+ qemuCapsProbeMachineTypes
+ qemuCapsProbeCPUModels
by splitting these functions into two parts. The existing function
name fetches the cached parse results for the specified binary and returns
them. The other half, named "qemuCapsCacheX", where X is one of
ExtractVersionInfo, ProbeMachineTypes, and ProbeCPUModels, exec's the
emulator binary and caches the results. The act of fetching the
cached results will fill or refresh the cache as necessary in a new
function qemuCapsCachedInfoGet(). A few auxilliary function have been
added -- e.g., virCapabilitiesDupMachines() to duplicate a cached list
of machine types and virBitmapDup() to duplicate cached capabilities
flags.
The series does not attempt to integrate with nor remove the existing
capabilities caching. TBD.
The series was developed and tested in the context of the Ubuntu 11.04 natty
libvirt_0.8.8-1ubuntu6.7 package using quilt to manage patches in the
debian/patches directory. In that context, it builds, passes all
"make check" tests [under pbuilder] and some fairly heavy, overlapping VM
launch tests where it does eliminate all but a few initial exec's of the
various qemu* and kvm binaries.
The version here, rebased to libvirt-0.9.10, builds cleanly under mock on
Fedora 16 in the context of a modified libvirt-0.9.10-1.fc16 source package.
I.e., no errors and warning-for-warning compatible with build of the
libvirt-0.9.10 fc16 srpm downloaded from libvirt.org. I placed the modified
spec file [applies the patches] and the build logs at:
http://downloads.linux.hp.com/~lts/Libvirt/
I have installed the patched libvirt on a fedora 16 system and successfully
defined and launched a vm. Testing in progress. I'll place an annotated
test log ont the site above when complete.
I also need to rebase atop the current mainline sources, but I wanted to get
this series out for review to see if the overall approach would be acceptable.
Comments?
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list