Re: [RFCv2] Introduce API for retrieving bulk domain stats v2

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Peter,

On 08/21/2014 09:20 PM, Peter Krempa wrote:
> I'd like to propose a (hopefully) fairly future-proof API to retrieve
> various statistics for domains.
> 
> The motivation is that management layers that use libvirt usually poll
> libvirt for statistics using various split up APIs we currently provide.
> To get all the necessary stuff, the mgmt app need to issue Ndomains *
> Napis calls and cope with the various returned formats. The APIs I'm
> wanting to introduce here will:
> 
> 1) Return data in a format that we can expand in the future and is
> hierarchical. This version returns the data as typed parameters where
> the fields are constructed as dot-separated strings containing names and
> other stuff in a list of typed params.
> 
> 2) Stats for multiple (all) domains can be queried at once and are
> returned in one call. This will allow to decrease the overhead necessary
> to issue multiple calls per domain multiplied by the count of domains.
> 
> 3) Selectable (bit mask) fields in the returned format. This will allow
> to retrieve only specific stats according to the APPs need.
> 
> Initially the implementation will introduce the option to retrieve
> block, interface  and cpu stats with the possibility to add more in the
> future.
> 
> The stats groups will be enabled using a bit field @stats passed as the
> function argument. A few groups for inspiration:
> 
> VIR_DOMAIN_STATS_STATE
> VIR_DOMAIN_STATS_CPU
> VIR_DOMAIN_STATS_BLOCK
> VIR_DOMAIN_STATS_INTERFACE

I'm working on an API to retrieve bulk domain block stats[1], after noticed
your "all-in-one" API for bulk stats retrieving I think your API would be
better for bulk stats retrieving.

I wonder if I could take the block part(VIR_DOMAIN_STATS_BLOCK) of this API?
If so, please cc me when you post the formal patchset so I can review it and
make the block part base on you patch.

> 
> the returned typed params will use the following scheme
> 
> state.state = running
> state.reason = started
> cpu.count = 8
> cpu.0.state = running
> cpu.0.time = 1234
> 
> ...
> 
> I'm also thinking of doing an async version of this, where the user
> would provide a callback that would be called for each returned domain
> but I'm not quite sure if that would play nicely with our RPC so that
> requires a bit more investigation first.

An async version would be useful to overcome the RPC message size limit
for the bulk stats.

[1]: https://www.redhat.com/archives/libvir-list/2014-August/msg00080.html

Thanks,
Li Wei


> 
> Peter
> ---
>  include/libvirt/libvirt.h.in |  26 +++++++
>  src/driver.h                 |   9 +++
>  src/libvirt.c                | 178 +++++++++++++++++++++++++++++++++++++++++++
>  src/libvirt_public.syms      |   7 ++
>  4 files changed, 220 insertions(+)
> 
> diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in
> index 47ea695..962f740 100644
> --- a/include/libvirt/libvirt.h.in
> +++ b/include/libvirt/libvirt.h.in
> @@ -2501,6 +2501,32 @@ int virDomainDetachDeviceFlags(virDomainPtr domain,
>  int virDomainUpdateDeviceFlags(virDomainPtr domain,
>                                 const char *xml, unsigned int flags);
> 
> +typedef struct _virDomainStatsRecord virDomainStatsRecord;
> +typedef virDomainStatsRecord *virDomainStatsRecordPtr;
> +struct _virDomainStatsRecord {
> +    virDomainPtr dom;
> +    unsigned int nparams;
> +    virTypedParameterPtr params;
> +};
> +
> +typedef enum {
> +    VIR_DOMAIN_STATS_ALL = (1 << 0), /* return all stats fields
> +                                       implemented in the daemon */
> +    VIR_DOMAIN_STATS_STATE = (1 << 1), /* return domain state */
> +} virDomainStatsTypes;
> +
> +int virConnectGetAllDomainStats(virConnectPtr conn,
> +                                unsigned int stats,
> +                                virDomainStatsRecordPtr **retStats,
> +                                unsigned int flags);
> +
> +int virDomainListGetStats(virDomainPtr *doms,
> +                          unsigned int stats,
> +                          virDomainStatsRecordPtr **retStats,
> +                          unsigned int flags);
> +
> +void virDomainStatsRecordListFree(virDomainStatsRecordPtr *stats);
> +
>  /*
>   * BlockJob API
>   */
> diff --git a/src/driver.h b/src/driver.h
> index 158df79..eab794d 100644
> --- a/src/driver.h
> +++ b/src/driver.h
> @@ -1191,6 +1191,14 @@ typedef int
>                            unsigned int flags);
> 
> 
> +typedef int
> +(*virDrvDomainListGetStats)(virConnectPtr conn,
> +                            virDomainPtr *doms,
> +                            unsigned int ndoms,
> +                            unsigned int stats,
> +                            virDomainStatsRecordPtr **retStats,
> +                            unsigned int flags);
> +
>  typedef struct _virDriver virDriver;
>  typedef virDriver *virDriverPtr;
> 
> @@ -1411,6 +1419,7 @@ struct _virDriver {
>      virDrvDomainSetTime domainSetTime;
>      virDrvNodeGetFreePages nodeGetFreePages;
>      virDrvConnectGetDomainCapabilities connectGetDomainCapabilities;
> +    virDrvDomainListGetStats domainListGetStats;
>  };
> 
> 
> diff --git a/src/libvirt.c b/src/libvirt.c
> index 8349261..c2244d0 100644
> --- a/src/libvirt.c
> +++ b/src/libvirt.c
> @@ -21341,3 +21341,181 @@ virConnectGetDomainCapabilities(virConnectPtr conn,
>      virDispatchError(conn);
>      return NULL;
>  }
> +
> +
> +/**
> + * virConnectGetAllDomainStats:
> + * @conn: pointer to the hypervisor connection
> + * @stats: stats to return, binary-OR of virDomainStatsTypes
> + * @retStats: Pointer that will be filled with the array of returned stats.
> + * @flags: extra flags; not used yet, so callers should always pass 0
> + *
> + * Query statistics for all domains on a given connection.
> + *
> + * Report statistics of various parameters for a running VM according to @stats
> + * field. The statistics are returned as an array of structures for each queried
> + * domain. The structure contains an array of typed parameters containing the
> + * individual statistics. The typed parameter name for each statistic field
> + * consists of a dot-separated string containing name of the requested group
> + * followed by a group specific description of the statistic value.
> + *
> + * The statistic groups are enabled using the @stats parameter which is a
> + * binary-OR of enum virDomainStatsTypes. The following groups are available
> + * (although not necessarily implemented for each storage driver):
> + *
> + * VIR_DOMAIN_STATS_ALL: Return all statistics supported by the hypervisor
> + * driver. This allows to query everything the driver supports without getting
> + * an error.
> + *
> + * VIR_DOMAIN_STATS_STATE: Return domain state and reason for entering that
> + * state. The typed parameter keys are in format:
> + * "state.state" - state of the VM, returned as int from virDomainState enum
> + * "state.reason" - reason for entering given state, returned as in from
> + *                  virDomain*Reason enmum corresponding to given state.
> + *
> + * Returns the count of returned statistics strucutres on success, -1 on error.
> + * The requested data are returned in the @retStats parameter. The returned
> + * array should be freed by the caller. See virDomainStatsRecordListFree.
> + */
> +int
> +virConnectGetAllDomainStats(virConnectPtr conn,
> +                            unsigned int stats,
> +                            virDomainStatsRecordPtr **retStats,
> +                            unsigned int flags)
> +{
> +    int ret = -1;
> +
> +    VIR_DEBUG("conn=%p, stats=0x%x, retStats=%p, flags=0x%x",
> +              conn, stats, retStats, flags);
> +
> +    virResetLastError();
> +
> +    virCheckConnectReturn(conn, -1);
> +
> +    if (!conn->driver->domainListGetStats) {
> +        virReportUnsupportedError();
> +        goto cleanup;
> +    }
> +
> +    ret = conn->driver->domainListGetStats(conn, NULL, 0, stats,
> +                                           retStats, flags);
> +
> + cleanup:
> +    if (ret < 0)
> +        virDispatchError(conn);
> +
> +    return ret;
> +}
> +
> +
> +/**
> + * virDomainListGetStats:
> + * @doms: NULL terminated array of domains
> + * @stats: stats to return, binary-OR of virDomainStatsTypes
> + * @retStats: Pointer that will be filled with the array of returned stats.
> + * @flags: extra flags; not used yet, so callers should always pass 0
> + *
> + * Query statistics for domains provided by @doms. Note that all domains in
> + * @doms must share the same connection.
> + *
> + * Report statistics of various parameters for a running VM according to @stats
> + * field. The statistics are returned as an array of structures for each queried
> + * domain. The structure contains an array of typed parameters containing the
> + * individual statistics. The typed parameter name for each statistic field
> + * consists of a dot-separated string containing name of the requested group
> + * followed by a group specific description of the statistic value.
> + *
> + * The statistic groups are enabled using the @stats parameter which is a
> + * binary-OR of enum virDomainStatsTypes. The following groups are available
> + * (although not necessarily implemented for each storage driver):
> + *
> + * VIR_DOMAIN_STATS_ALL: Return all statistics supported by the hypervisor
> + * driver. This allows to query everything the driver supports without getting
> + * an error.
> + *
> + * VIR_DOMAIN_STATS_STATE: Return domain state and reason for entering that
> + * state. The typed parameter keys are in format:
> + * "state.state" - state of the VM, returned as int from virDomainState enum
> + * "state.reason" - reason for entering given state, returned as in from
> + *                  virDomain*Reason enmum corresponding to given state.
> + *
> + * Returns the count of returned statistics strucutres on success, -1 on error.
> + * The requested data are returned in the @retStats parameter. The returned
> + * array should be freed by the caller. See virDomainStatsRecordListFree.
> + */
> +int
> +virDomainListGetStats(virDomainPtr *doms,
> +                      unsigned int stats,
> +                      virDomainStatsRecordPtr **retStats,
> +                      unsigned int flags)
> +{
> +    virConnectPtr conn = NULL;
> +    virDomainPtr *nextdom = doms;
> +    unsigned int ndoms = 0;
> +    int ret = -1;
> +
> +    VIR_DEBUG("doms=%p, stats=0x%x, retStats=%p, flags=0x%x",
> +              doms, stats, retStats, flags);
> +
> +    virResetLastError();
> +
> +    if (!*doms) {
> +        virReportError(VIR_ERR_INVALID_ARG,
> +                       _("doms array in %s must contain at leas one domain"),
> +                       __FUNCTION__);
> +        goto cleanup;
> +    }
> +
> +    conn = doms[0]->conn;
> +
> +    if (!conn->driver->domainListGetStats) {
> +        virReportUnsupportedError();
> +        goto cleanup;
> +    }
> +
> +    while (*(++nextdom)) {
> +        virDomainPtr dom = *nextdom;
> +
> +        virCheckDomainGoto(dom, cleanup);
> +
> +        if (dom->conn != conn) {
> +            virReportError(VIR_ERR_INVALID_ARG,
> +                           _("domains in 'doms' array must belong to a "
> +                             "single connection in %s"), __FUNCTION__);
> +            goto cleanup;
> +        }
> +
> +        ndoms++;
> +    }
> +
> +    ret = conn->driver->domainListGetStats(conn, doms, ndoms, stats, retStats,
> +                                           flags);
> +
> + cleanup:
> +    if (ret < 0)
> +        virDispatchError(conn);
> +    return ret;
> +}
> +
> +
> +/**
> + * virDomainStatsRecordListFree:
> + * @stats: NULL terminated array of virDomainStatsRecords to free
> + *
> + * Convenience function to free a list of domain stats returned by
> + * virDomainListGetStats and virConnectGetAllDomainStats.
> + */
> +void
> +virDomainStatsRecordListFree(virDomainStatsRecordPtr *stats)
> +{
> +    virDomainStatsRecordPtr *next = stats;
> +
> +    while (*next) {
> +        virTypedParamsFree((*next)->params, (*next)->nparams);
> +        virDomainFree((*next)->dom);
> +        VIR_FREE((*next));
> +        next++;
> +    }
> +
> +    VIR_FREE(stats);
> +}
> diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms
> index 9f4016a..3714159 100644
> --- a/src/libvirt_public.syms
> +++ b/src/libvirt_public.syms
> @@ -670,4 +670,11 @@ LIBVIRT_1.2.7 {
>          virConnectGetDomainCapabilities;
>  } LIBVIRT_1.2.6;
> 
> +LIBVIRT_1.2.8 {
> +    global:
> +        virDomainListGetStats;
> +        virConnectGetAllDomainStats;
> +        virDomainStatsRecordListFree;
> +} LIBVIRT_1.2.7;
> +
>  # .... define new API here using predicted next version number ....
> 

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]