On Mon, 2018-11-26 at 17:36 +0200, Ariel Almog wrote:
> Hi,
>
> Please find below, an RFC for exposing statistics counters for RDMA subsystem.
>
> The proposed tool, will be a supplementary command to already existed
> rdma tool and will give user a variety of options for RDMA debug and
> performance analysis
>
> The tool will provide out of the box global counters and in addition, user will
> be able to manually tune it to monitor specific QP(s) or automatically monitor
> QP(s) according to predefined criteria
>
> In details
> **********
>
> In this RFC, the term "counter set" refers to a set of multiple counters
> that builds a set of counters. For example, a counter set might include
> duplicate request counter, implied nak seq err counter, etc.
>
> A counter set is part of namespace and cannot be viewed/set from other
> namespaces.
>
> 1. Default, general counter
> ***************************
> This is a optional, default, counter set, which is allocated by the RDMA
> driver upon init and is used to count all the traffic that is passing through
> the device. No configuration is needed and no disablement is available
>
> Usage:
> rdma dev stat
> Shows the statistics of the the general counter set
>
> 2. Manual bind/unbind of counter set to QP(s)
> *********************************************
> This is an optional interface to allow user to select QPs to be monitored.
> The process require the user to
> a. Allocate counter_set
> b. Bind QP(s) to the allocated counter_set
> c. Monitor the counter_set
> d. Unbind QP(s) from the allocated counter_set
> e. Deallocate the counter_set
>
> Usage:
> rdma dev stat alloc
> Allocates and returns a counter set id which can be bound to QP(s)
>
> rdma dev stat dealloc <counter set id>
> Deallocates a counter set id. all bounded QP(s) shall be unbind before
> deallocation
>
> rdma dev stat bind <qp num> <counter set id>
> Binds QP to a counter set
>
> rdma dev stat unbind <qp num>
> Unbinds a QP from a counter set
>
>
> 3. Automatic bind/unbind of counter sets to QPs
> ***********************************************
> This is an optional interface to allow user to build automatic sets of QPs to
> counter set according to common criteria. For example a per pid scheme, where
> all QPs belong to pid are bind automatically to a single counter set.
>
> Usage:
> rdma dev stat auto pid
> Allocates counter set per pid, bind all pid's QPs to this counter set
>
> rdma dev stat auto type
> Allocates counter set per QP type (RC, UD, Raw, etc., bind all QPs of this
> type to this counter set
Combinations of the types would be nice. So rdma dev stat auto pid auto
type would give you stats on the different types of QPs just for a
specific pid. Doing this on a process tree would allow you to capture
accounting for child processes along with the parent once the fork
support that Parav mentioned is in the kernel. A process tree would
also trivially allow you to capture statistics for a cgroup by running
it on the parent pid of the cgroup.
> rdma dev stat auto off
> Deallocates all counter set that were configured as auto
>
> 4. Exposing of counters set content
> ***********************************
> This is an optional interface to print the counter set content. it can print
> all counter sets or specific counter set according to the working mode.
>
> Usage:
> rdma dev stat show [counter set id]|[pid]|[type]
> Print statistics counters.
> - In case manual mode was used, optional <counter set id> can be used
> - In case auto per pid mode was used, optional <pid> can be used
> - In case auto per type mode was used, optional <type> can be used
>
You probably want to require the explicit use of the words id, pid, and
type when displaying the stats so there is no ambiguity. It also allows
you to specify more than one in the case that you did as I suggest above
and create, say, per pid and per type stats, you could then ask for the
stats of the pid and get all of the stats for all types back, or specify
the pid and some type to get just a single one.
> I would like to get feedback on this proposal
>
> Thanks
> Ariel Almog
> Mellanox
--
Doug Ledford <dledford@xxxxxxxxxx>
GPG KeyID: B826A3330E572FDD
Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
Attachment:
signature.asc
Description: This is a digitally signed message part
