On Fri, Jul 07, 2006 at 03:47:59PM +0100, Daniel P. Berrange wrote:
> Attached is a patch to significantly increase scalability / performance of
> the xenDaemonLookupByID method. The current implementation would get a
> list of all domain names from XenD, and then iterate doing a HTTP GET on
> /xend/domain/[name] until the domain with match ID was found. THis had
> O(n) complexity, with the result that when running on a system with 20
> actives domains, 'virsh list' would have O(n^2) complexity needing ~230
> HTTP calls, giving a runtime of ~9 seconds.
>
> The patch is to make the code do a HTTP GET on /xend/domain/[id] which we
> just discovered is a valid URL to access. This makes the method call O(1),
> and 'virsh list' is now a saner O(n), and completes in ~1 second. While
> still not great performance, this is certainly much better. I think it
> ought to be possible to optimize the code still further so that XenD is
> avoided altogether for simple commands which can be fullfilled purely
> with data available from Hypervisor, but that will need further
> investigation.
i
So my previous post considered performance when doing a single iteration
over all active domains. I've now moved onto look at what happens when
you do multiple iterations - eg a monitoring application sampling state
every 5 seconds.
Take code which connects to the HV, and then periodically lists all domains
and gets their status, eg
conn = virConnectOpen(NULL);
for (j = 0 ; j < 5 ; j++) {
nid = virConnectListDomains(conn, ids, sizeof(ids)/sizeof(int));
for (i = 0 ; i < nid ; i++) {
int id = ids[i];
dom = virDomainLookupByID(conn, id);
ret = virDomainGetInfo(dom, &info);
printf("Domains %d: %d CPUs\n", id, info.nrVirtCpu);
}
}
Then, every single call to virDomainLookupByID will trigger an HTTP GET
to resolve id -> name,uuid. Now the virConnect data structure contains
a hash table storing all known domains, however, the xenDaemonLookupByID
method only uses the cache once its done the HTTP GET to resolv id -> name.
This is obviously needed the first time around, however, subsequent calls
to xenDaemonLookupByID have no need to do the HTTP GET since the cache
already contains a virDomain instance with id, name, uuid resolved.
If you measure the above code snippet on a machine with 20 domains, then
5 iterations of the outer loop will require 101 HTTP GET's and take about
5 seconds to complete.
If we modify the xenDaemonLookupByID method to check the cache immediately,
doing a cache lookup on ID, rather than name, we only need to do the HTTP
GET request the very first time a domain is looked up. Thus, only the first
iteration of the outer loop will result in HTTP calls - you can do as many
iterations as you like and there will only ever by 21 HTTP GETs performed,
and total runtime for 5 iterations drops to 1 second. The first iteration
does HTTP GETS to resolve name & uuid, but all subsquent iterations only
need to do HyperCalls so are lightening fast.
I'm attaching a patch which implements this speed up. The only issue not
being dealt with here is cache eviction. Xen Domain IDs will always increase,
no ID is re-used until the IDs wrap around - I can't remember whether the
ID space is 16 or 32-bits, but even with 16bits we'd need 32,000 domains to
have been created before wrap-around becomes an issue. We'd need to address
it long term, but for now I think we can live with this. Cache evication
could probaly be done in the virListDomains method - ie evict any entries
with IDs not currently running
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
diff -ru libvirt-new/src/proxy_internal.c libvirt/src/proxy_internal.c
--- libvirt-new/src/proxy_internal.c 2006-07-03 11:48:49.000000000 -0400
+++ libvirt/src/proxy_internal.c 2006-07-10 08:51:18.000000000 -0400
@@ -760,6 +760,11 @@
virProxyError(conn, VIR_ERR_INVALID_ARG, __FUNCTION__);
return (NULL);
}
+
+ if ((res = virGetDomainByID(conn, id)) != NULL) {
+ return res;
+ }
+
memset(&req, 0, sizeof(req));
req.command = VIR_PROXY_LOOKUP_ID;
req.data.arg = id;
diff -ru libvirt-new/src/xend_internal.c libvirt/src/xend_internal.c
--- libvirt-new/src/xend_internal.c 2006-07-10 09:00:15.000000000 -0400
+++ libvirt/src/xend_internal.c 2006-07-10 08:49:42.000000000 -0400
@@ -2403,6 +2403,10 @@
unsigned char uuid[16];
virDomainPtr ret;
+ if ((ret = virGetDomainByID(conn, id)) != NULL) {
+ return ret;
+ }
+
if (xenDaemonDomainLookupByID(conn, id, &name, uuid) < 0) {
goto error;
}