On Mon, Jun 14, 2021 at 09:21:53PM +0000, Jing Zhang wrote: >Update KVM API documentation for binary statistics.You should write more here. See my comment at the bottom... >>Reviewed-by: David Matlack <dmatlack@xxxxxxxxxx>>Reviewed-by: Ricardo Koller <ricarkol@xxxxxxxxxx>>Reviewed-by: Krish Sadhukhan <krish.sadhukhan@xxxxxxxxxx>>Reviewed-by: Fuad Tabba <tabba@xxxxxxxxxx>>Signed-off-by: Jing Zhang <jingzhangos@xxxxxxxxxx>>--->Documentation/virt/kvm/api.rst | 177 ++++++++++++++++++++++++++++++++->1 file changed, 176 insertions(+), 1 deletion(-)>>diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst>index e328caa35d6c..35ee52dbec89 100644>--- a/Documentation/virt/kvm/api.rst>+++ b/Documentation/virt/kvm/api.rst>@@ -5034,7 +5034,6 @@ see KVM_XEN_VCPU_SET_ATTR above.>The KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADJUST type may not be used>with the KVM_XEN_VCPU_GET_ATTR ioctl.>>->4.131 KVM_GET_SREGS2>------------------>>@@ -5081,6 +5080,174 @@ Writes special registers into the vcpu.>See KVM_GET_SREGS2 for the data structures.>This ioctl (when supported) replaces the KVM_SET_SREGS.>>+4.133 KVM_GET_STATS_FD>+---------------------->+>+:Capability: KVM_CAP_STATS_BINARY_FD>+:Architectures: all>+:Type: vm ioctl, vcpu ioctl>+:Parameters: none>+:Returns: statistics file descriptor on success, < 0 on error>+>+Errors:>+>+ ====== ======================================================>+ ENOMEM if the fd could not be created due to lack of memory>+ EMFILE if the number of opened files exceeds the limit>+ ====== ======================================================>+>+The file descriptor can be used to read VM/vCPU statistics data in binary>+format. The file data is organized into three blocks as below:>++-------------+>+| Header |>++-------------+>+| Descriptors |>++-------------+>+| Stats Data |>++-------------+>+>+The Header block is always at the start of the file. It is only needed to be>+read one time for the lifetime of the file descriptor.>+It is in the form of ``struct kvm_stats_header`` as below::>+>+ #define KVM_STATS_ID_MAXLEN 64>+>+ struct kvm_stats_header {>+ __u32 name_size;>+ __u32 count;>+ __u32 desc_offset;>+ __u32 data_offset;>+ char id[0];>+ };>+>+The ``id`` field is identification for the corresponding KVM statistics. For>+VM statistics, it is in the form of "kvm-{kvm pid}", like "kvm-12345". For>+VCPU statistics, it is in the form of "kvm-{kvm pid}/vcpu-{vcpu id}", like>+"kvm-12345/vcpu-12".>+>+The ``name_size`` field is the size (byte) of the statistics name string>+(including trailing '\0') appended to the end of every statistics descriptor.>+>+The ``count`` field is the number of statistics.>+>+The ``desc_offset`` field is the offset of the Descriptors block from the start>+of the file indicated by the file descriptor.>+>+The ``data_offset`` field is the offset of the Stats Data block from the start>+of the file indicated by the file descriptor.>+>+The Descriptors block is only needed to be read once for the lifetime of the>+file descriptor. It is an array of ``struct kvm_stats_desc`` as shown in>+below code block::>+>+ #define KVM_STATS_TYPE_SHIFT 0>+ #define KVM_STATS_TYPE_MASK (0xF << KVM_STATS_TYPE_SHIFT)>+ #define KVM_STATS_TYPE_CUMULATIVE (0x0 << KVM_STATS_TYPE_SHIFT)>+ #define KVM_STATS_TYPE_INSTANT (0x1 << KVM_STATS_TYPE_SHIFT)>+ #define KVM_STATS_TYPE_MAX KVM_STATS_TYPE_INSTANT>+>+ #define KVM_STATS_UNIT_SHIFT 4>+ #define KVM_STATS_UNIT_MASK (0xF << KVM_STATS_UNIT_SHIFT)>+ #define KVM_STATS_UNIT_NONE (0x0 << KVM_STATS_UNIT_SHIFT)>+ #define KVM_STATS_UNIT_BYTES (0x1 << KVM_STATS_UNIT_SHIFT)>+ #define KVM_STATS_UNIT_SECONDS (0x2 << KVM_STATS_UNIT_SHIFT)>+ #define KVM_STATS_UNIT_CYCLES (0x3 << KVM_STATS_UNIT_SHIFT)>+ #define KVM_STATS_UNIT_MAX KVM_STATS_UNIT_CYCLES>+>+ #define KVM_STATS_BASE_SHIFT 8>+ #define KVM_STATS_BASE_MASK (0xF << KVM_STATS_BASE_SHIFT)>+ #define KVM_STATS_BASE_POW10 (0x0 << KVM_STATS_BASE_SHIFT)>+ #define KVM_STATS_BASE_POW2 (0x1 << KVM_STATS_BASE_SHIFT)>+ #define KVM_STATS_BASE_MAX KVM_STATS_BASE_POW2>+>+ struct kvm_stats_desc {>+ __u32 flags;>+ __s16 exponent;>+ __u16 size;>+ __u32 offset;>+ __u32 unused;>+ char name[0];>+ };>+>+The ``flags`` field contains the type and unit of the statistics data described>+by this descriptor. The following flags are supported:>+>+Bits 0-3 of ``flags`` encode the type:>+ * ``KVM_STATS_TYPE_CUMULATIVE``>+ The statistics data is cumulative. The value of data can only be increased.>+ Most of the counters used in KVM are of this type.>+ The corresponding ``count`` field for this type is always 1.>+ * ``KVM_STATS_TYPE_INSTANT``>+ The statistics data is instantaneous. Its value can be increased or>+ decreased. This type is usually used as a measurement of some resources,>+ like the number of dirty pages, the number of large pages, etc.>+ The corresponding ``count`` field for this type is always 1.>+>+Bits 4-7 of ``flags`` encode the unit:>+ * ``KVM_STATS_UNIT_NONE``>+ There is no unit for the value of statistics data. This usually means that>+ the value is a simple counter of an event.>+ * ``KVM_STATS_UNIT_BYTES``>+ It indicates that the statistics data is used to measure memory size, in the>+ unit of Byte, KiByte, MiByte, GiByte, etc. The unit of the data is>+ determined by the ``exponent`` field in the descriptor. The>+ ``KVM_STATS_BASE_POW2`` flag is valid in this case. The unit of the data is>+ determined by ``pow(2, exponent)``. For example, if value is 10,>+ ``exponent`` is 20, which means the unit of statistics data is MiByte, we>+ can get the statistics data in the unit of Byte by>+ ``value * pow(2, exponent) = 10 * pow(2, 20) = 10 MiByte`` which is>+ 10 * 1024 * 1024 Bytes.>+ * ``KVM_STATS_UNIT_SECONDS``>+ It indicates that the statistics data is used to measure time/latency, in>+ the unit of nanosecond, microsecond, millisecond and second. The unit of the>+ data is determined by the ``exponent`` field in the descriptor. The>+ ``KVM_STATS_BASE_POW10`` flag is valid in this case. The unit of the data>+ is determined by ``pow(10, exponent)``. For example, if value is 2000000,>+ ``exponent`` is -6, which means the unit of statistics data is microsecond,>+ we can get the statistics data in the unit of second by>+ ``value * pow(10, exponent) = 2000000 * pow(10, -6) = 2 seconds``.>+ * ``KVM_STATS_UNIT_CYCLES``>+ It indicates that the statistics data is used to measure CPU clock cycles.>+ The ``KVM_STATS_BASE_POW10`` flag is valid in this case. For example, if>+ value is 200, ``exponent`` is 4, we can get the number of CPU clock cycles>+ by ``value * pow(10, exponent) = 200 * pow(10, 4) = 2000000``.>+>+Bits 8-11 of ``flags`` encode the base:>+ * ``KVM_STATS_BASE_POW10``>+ The scale is based on power of 10. It is used for measurement of time and>+ CPU clock cycles.>+ * ``KVM_STATS_BASE_POW2``>+ The scale is based on power of 2. It is used for measurement of memory size.>+>+The ``exponent`` field is the scale of corresponding statistics data. For>+example, if the unit is ``KVM_STATS_UNIT_BYTES``, the base is>+``KVM_STATS_BASE_POW2``, the ``exponent`` is 10, then we know that the real>+unit of the statistics data is KBytes a.k.a pow(2, 10) = 1024 bytes.>+>+The ``size`` field is the number of values (u64) of this statistics data. Its>+value is usually 1 for most of simple statistics.>+>+The ``offset`` field is the offset from the start of Data Block to the start of>+the corresponding statistics data.>+>+The ``unused`` fields are reserved for future support for other types of>+statistics data, like log/linear histogram.>+>+The ``name`` field points to the name string of the statistics data. The name>+string starts at the end of ``struct kvm_stats_desc``.>+The maximum length (including trailing '\0') is indicated by ``name_size``>+in ``struct kvm_stats_header``.>+>+The Stats Data block contains an array of data values of type ``struct>+kvm_vm_stats_data`` or ``struct kvm_vcpu_stats_data``. It would be read by>+userspace periodically to pull statistics data.>+The order of data value in Stats Data block is the same as the order of>+descriptors in Descriptors block.>+ * Statistics data for VM/VCPU::>+>+ struct kvm_stats_data {>+ __u64 value[0];>+ };I forgot to comment on this one, sorry for the delay. Why are you "inventing" your own schema format here for this? Why not use a well-known or at least well-designed/implemented one that we have in userspace already? There are a few that I would love to see in the kernel, varlink being the best example. We have kernel examples of this and I would consider using that as a transport for sysfs-like data in the future, but never got around to it. So again, why reinvent the wheel to create a custom api when you could use an existing one? thanks, greg k-h