On Fri, Jul 04, 2014 at 12:33:27PM +0100, Daniel P. Berrange wrote: > On Fri, Jul 04, 2014 at 12:11:54PM +0100, Richard W.M. Jones wrote: > > On Tue, Jul 01, 2014 at 03:09:13AM -0400, Francesco Romani wrote: > > > I'd like to discuss possible APIs and plans for new query APIs in libvirt. > > > > > > I'm one of the oVirt (http://www.ovirt.org) developers, and I write code for VDSM; > > > VDSM is the node management daemon, which is in charge, among many other things, to > > > gather the host and statistics per Domain/VM. > > > > > > Right now we aim for a number of VM per node in the (few) hundreds, but we have big plans > > > to scale much more, and to possibly reach thousands in a not so distant future. > > > At the moment, we use one thread per VM to gather the VM stats (CPU, network, disk), > > > and of course this obviously scales poorly. > > > > I'll just note here that a bug has been opened for virt-top, which > > is similar to this. > > > > If a domain has a large number of disks (256 virtio-scsi disks in the > > customer's case), then virt-top spends so long fetching the data for > > each separate disk, it can take 30-40 seconds between updates. > > > > The same thing would happen if you had lots of domains, each with a > > few disks, but with the total adding up to hundreds of disks. > > > > The same thing would happen if you substitute network interfaces for disks. > > > > What would help for us: > > > > - A way to get information for multiple objects in a single domain > > > > - A way to get information for multiple objects across multiple domains > > I'd say that we want something similar to the virDomainListAllDomains() > API for stats. ie we shouldn't try to pass in the full list of domains > or paths we want info for. We should just list all domains, optionally > using flags to filter based on some characteristic, eg exclude inactive. > Similarly always list stats for all disks. FYI for virt-top we only care about stats of all active domains, and we only care about all disks & all network interfaces for domains (ie. never any subset). We also collect CPU time and memory usage per domain. Of course this only applies to virt-top, not to other clients. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list