Hi Peter, On 08/21/2014 09:20 PM, Peter Krempa wrote: > I'd like to propose a (hopefully) fairly future-proof API to retrieve > various statistics for domains. > > The motivation is that management layers that use libvirt usually poll > libvirt for statistics using various split up APIs we currently provide. > To get all the necessary stuff, the mgmt app need to issue Ndomains * > Napis calls and cope with the various returned formats. The APIs I'm > wanting to introduce here will: > > 1) Return data in a format that we can expand in the future and is > hierarchical. This version returns the data as typed parameters where > the fields are constructed as dot-separated strings containing names and > other stuff in a list of typed params. > > 2) Stats for multiple (all) domains can be queried at once and are > returned in one call. This will allow to decrease the overhead necessary > to issue multiple calls per domain multiplied by the count of domains. > > 3) Selectable (bit mask) fields in the returned format. This will allow > to retrieve only specific stats according to the APPs need. > > Initially the implementation will introduce the option to retrieve > block, interface and cpu stats with the possibility to add more in the > future. > > The stats groups will be enabled using a bit field @stats passed as the > function argument. A few groups for inspiration: > > VIR_DOMAIN_STATS_STATE > VIR_DOMAIN_STATS_CPU > VIR_DOMAIN_STATS_BLOCK > VIR_DOMAIN_STATS_INTERFACE I'm working on an API to retrieve bulk domain block stats[1], after noticed your "all-in-one" API for bulk stats retrieving I think your API would be better for bulk stats retrieving. I wonder if I could take the block part(VIR_DOMAIN_STATS_BLOCK) of this API? If so, please cc me when you post the formal patchset so I can review it and make the block part base on you patch. > > the returned typed params will use the following scheme > > state.state = running > state.reason = started > cpu.count = 8 > cpu.0.state = running > cpu.0.time = 1234 > > ... > > I'm also thinking of doing an async version of this, where the user > would provide a callback that would be called for each returned domain > but I'm not quite sure if that would play nicely with our RPC so that > requires a bit more investigation first. An async version would be useful to overcome the RPC message size limit for the bulk stats. [1]: https://www.redhat.com/archives/libvir-list/2014-August/msg00080.html Thanks, Li Wei > > Peter > --- > include/libvirt/libvirt.h.in | 26 +++++++ > src/driver.h | 9 +++ > src/libvirt.c | 178 +++++++++++++++++++++++++++++++++++++++++++ > src/libvirt_public.syms | 7 ++ > 4 files changed, 220 insertions(+) > > diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in > index 47ea695..962f740 100644 > --- a/include/libvirt/libvirt.h.in > +++ b/include/libvirt/libvirt.h.in > @@ -2501,6 +2501,32 @@ int virDomainDetachDeviceFlags(virDomainPtr domain, > int virDomainUpdateDeviceFlags(virDomainPtr domain, > const char *xml, unsigned int flags); > > +typedef struct _virDomainStatsRecord virDomainStatsRecord; > +typedef virDomainStatsRecord *virDomainStatsRecordPtr; > +struct _virDomainStatsRecord { > + virDomainPtr dom; > + unsigned int nparams; > + virTypedParameterPtr params; > +}; > + > +typedef enum { > + VIR_DOMAIN_STATS_ALL = (1 << 0), /* return all stats fields > + implemented in the daemon */ > + VIR_DOMAIN_STATS_STATE = (1 << 1), /* return domain state */ > +} virDomainStatsTypes; > + > +int virConnectGetAllDomainStats(virConnectPtr conn, > + unsigned int stats, > + virDomainStatsRecordPtr **retStats, > + unsigned int flags); > + > +int virDomainListGetStats(virDomainPtr *doms, > + unsigned int stats, > + virDomainStatsRecordPtr **retStats, > + unsigned int flags); > + > +void virDomainStatsRecordListFree(virDomainStatsRecordPtr *stats); > + > /* > * BlockJob API > */ > diff --git a/src/driver.h b/src/driver.h > index 158df79..eab794d 100644 > --- a/src/driver.h > +++ b/src/driver.h > @@ -1191,6 +1191,14 @@ typedef int > unsigned int flags); > > > +typedef int > +(*virDrvDomainListGetStats)(virConnectPtr conn, > + virDomainPtr *doms, > + unsigned int ndoms, > + unsigned int stats, > + virDomainStatsRecordPtr **retStats, > + unsigned int flags); > + > typedef struct _virDriver virDriver; > typedef virDriver *virDriverPtr; > > @@ -1411,6 +1419,7 @@ struct _virDriver { > virDrvDomainSetTime domainSetTime; > virDrvNodeGetFreePages nodeGetFreePages; > virDrvConnectGetDomainCapabilities connectGetDomainCapabilities; > + virDrvDomainListGetStats domainListGetStats; > }; > > > diff --git a/src/libvirt.c b/src/libvirt.c > index 8349261..c2244d0 100644 > --- a/src/libvirt.c > +++ b/src/libvirt.c > @@ -21341,3 +21341,181 @@ virConnectGetDomainCapabilities(virConnectPtr conn, > virDispatchError(conn); > return NULL; > } > + > + > +/** > + * virConnectGetAllDomainStats: > + * @conn: pointer to the hypervisor connection > + * @stats: stats to return, binary-OR of virDomainStatsTypes > + * @retStats: Pointer that will be filled with the array of returned stats. > + * @flags: extra flags; not used yet, so callers should always pass 0 > + * > + * Query statistics for all domains on a given connection. > + * > + * Report statistics of various parameters for a running VM according to @stats > + * field. The statistics are returned as an array of structures for each queried > + * domain. The structure contains an array of typed parameters containing the > + * individual statistics. The typed parameter name for each statistic field > + * consists of a dot-separated string containing name of the requested group > + * followed by a group specific description of the statistic value. > + * > + * The statistic groups are enabled using the @stats parameter which is a > + * binary-OR of enum virDomainStatsTypes. The following groups are available > + * (although not necessarily implemented for each storage driver): > + * > + * VIR_DOMAIN_STATS_ALL: Return all statistics supported by the hypervisor > + * driver. This allows to query everything the driver supports without getting > + * an error. > + * > + * VIR_DOMAIN_STATS_STATE: Return domain state and reason for entering that > + * state. The typed parameter keys are in format: > + * "state.state" - state of the VM, returned as int from virDomainState enum > + * "state.reason" - reason for entering given state, returned as in from > + * virDomain*Reason enmum corresponding to given state. > + * > + * Returns the count of returned statistics strucutres on success, -1 on error. > + * The requested data are returned in the @retStats parameter. The returned > + * array should be freed by the caller. See virDomainStatsRecordListFree. > + */ > +int > +virConnectGetAllDomainStats(virConnectPtr conn, > + unsigned int stats, > + virDomainStatsRecordPtr **retStats, > + unsigned int flags) > +{ > + int ret = -1; > + > + VIR_DEBUG("conn=%p, stats=0x%x, retStats=%p, flags=0x%x", > + conn, stats, retStats, flags); > + > + virResetLastError(); > + > + virCheckConnectReturn(conn, -1); > + > + if (!conn->driver->domainListGetStats) { > + virReportUnsupportedError(); > + goto cleanup; > + } > + > + ret = conn->driver->domainListGetStats(conn, NULL, 0, stats, > + retStats, flags); > + > + cleanup: > + if (ret < 0) > + virDispatchError(conn); > + > + return ret; > +} > + > + > +/** > + * virDomainListGetStats: > + * @doms: NULL terminated array of domains > + * @stats: stats to return, binary-OR of virDomainStatsTypes > + * @retStats: Pointer that will be filled with the array of returned stats. > + * @flags: extra flags; not used yet, so callers should always pass 0 > + * > + * Query statistics for domains provided by @doms. Note that all domains in > + * @doms must share the same connection. > + * > + * Report statistics of various parameters for a running VM according to @stats > + * field. The statistics are returned as an array of structures for each queried > + * domain. The structure contains an array of typed parameters containing the > + * individual statistics. The typed parameter name for each statistic field > + * consists of a dot-separated string containing name of the requested group > + * followed by a group specific description of the statistic value. > + * > + * The statistic groups are enabled using the @stats parameter which is a > + * binary-OR of enum virDomainStatsTypes. The following groups are available > + * (although not necessarily implemented for each storage driver): > + * > + * VIR_DOMAIN_STATS_ALL: Return all statistics supported by the hypervisor > + * driver. This allows to query everything the driver supports without getting > + * an error. > + * > + * VIR_DOMAIN_STATS_STATE: Return domain state and reason for entering that > + * state. The typed parameter keys are in format: > + * "state.state" - state of the VM, returned as int from virDomainState enum > + * "state.reason" - reason for entering given state, returned as in from > + * virDomain*Reason enmum corresponding to given state. > + * > + * Returns the count of returned statistics strucutres on success, -1 on error. > + * The requested data are returned in the @retStats parameter. The returned > + * array should be freed by the caller. See virDomainStatsRecordListFree. > + */ > +int > +virDomainListGetStats(virDomainPtr *doms, > + unsigned int stats, > + virDomainStatsRecordPtr **retStats, > + unsigned int flags) > +{ > + virConnectPtr conn = NULL; > + virDomainPtr *nextdom = doms; > + unsigned int ndoms = 0; > + int ret = -1; > + > + VIR_DEBUG("doms=%p, stats=0x%x, retStats=%p, flags=0x%x", > + doms, stats, retStats, flags); > + > + virResetLastError(); > + > + if (!*doms) { > + virReportError(VIR_ERR_INVALID_ARG, > + _("doms array in %s must contain at leas one domain"), > + __FUNCTION__); > + goto cleanup; > + } > + > + conn = doms[0]->conn; > + > + if (!conn->driver->domainListGetStats) { > + virReportUnsupportedError(); > + goto cleanup; > + } > + > + while (*(++nextdom)) { > + virDomainPtr dom = *nextdom; > + > + virCheckDomainGoto(dom, cleanup); > + > + if (dom->conn != conn) { > + virReportError(VIR_ERR_INVALID_ARG, > + _("domains in 'doms' array must belong to a " > + "single connection in %s"), __FUNCTION__); > + goto cleanup; > + } > + > + ndoms++; > + } > + > + ret = conn->driver->domainListGetStats(conn, doms, ndoms, stats, retStats, > + flags); > + > + cleanup: > + if (ret < 0) > + virDispatchError(conn); > + return ret; > +} > + > + > +/** > + * virDomainStatsRecordListFree: > + * @stats: NULL terminated array of virDomainStatsRecords to free > + * > + * Convenience function to free a list of domain stats returned by > + * virDomainListGetStats and virConnectGetAllDomainStats. > + */ > +void > +virDomainStatsRecordListFree(virDomainStatsRecordPtr *stats) > +{ > + virDomainStatsRecordPtr *next = stats; > + > + while (*next) { > + virTypedParamsFree((*next)->params, (*next)->nparams); > + virDomainFree((*next)->dom); > + VIR_FREE((*next)); > + next++; > + } > + > + VIR_FREE(stats); > +} > diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms > index 9f4016a..3714159 100644 > --- a/src/libvirt_public.syms > +++ b/src/libvirt_public.syms > @@ -670,4 +670,11 @@ LIBVIRT_1.2.7 { > virConnectGetDomainCapabilities; > } LIBVIRT_1.2.6; > > +LIBVIRT_1.2.8 { > + global: > + virDomainListGetStats; > + virConnectGetAllDomainStats; > + virDomainStatsRecordListFree; > +} LIBVIRT_1.2.7; > + > # .... define new API here using predicted next version number .... > -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list