On Mon, Apr 01, 2019 at 02:59:16PM +0300, Gal Pressman wrote: > On 01-Apr-19 11:47, Leon Romanovsky wrote: > > From: Leon Romanovsky <leonro@xxxxxxxxxxxx> > > > > Hi, > > > > This series from Mark provides dynamic statistics infrastructure. > > He uses netlink interface to configure and retrieve those counters. > > > > This infrastructure allows to users monitor various objects by binding > > to them counters. As the beginning, we used QP object as target for > > those counters, but future patches will include ODP MR information too. > > Hi Leon and Mark, > Thanks for doing this! > > > > > Two binding modes are supported: > > - Auto: This allows a user to build automatic set of objects to a counter > > build = bind? "build" == "chain". In theory, user will be able to create very complex filters, send those chains and kernel will handle it. For example, bind counters for UD QP, on specific port and for new processes all together. > > > according to common criteria. For example in a per-type scheme, where in > > one process all QPs with same QP type are bound automatically to a single > > counter. > > How do we decide which criteria is suitable for auto mode and why is it better > than letting the userspace handle it by itself (query all QPs and bind certain > types to "manual" counters). > Seems like doing it in userspace provides more flexibility than a fixed set of > kernel auto types. "Auto mode" allows to get counters during object creation, for example in ODP MR case, it will give us a chance to count pagefaults immediately after MRs are created. It is good from system perspective too, he will need to configure policy only once during boot and it will simply work. > > Is there a reason to have one auto counter per port? > Theoretically I can allocate two auto counters and assign a different auto mask > to each one. Sometimes you need to say enough is enough :), we didn't want to add so much complexity without solid use case justification. From implementation perspective, we will be able to do it later, because it won't require any change in kernel API. Just need to ensure that such masks are returned with dumpit. > > > - Manual: This allows a user to manually bind objects on a counter. > > > > Those two modes are mutual-exclusive with separation between processes, > > objects created by different processes cannot be bound to a same counter. > > > > For objects which don't support counter binding, we will return > > pre-allocated counters. > > Can you explain? What are those objects and what are pre allocated counters? For example MR counters, we thought to add very simple set of them and make always available. > > > > > $ rdma statistic qp set link mlx5_2/1 auto type on > > $ rdma statistic qp set link mlx5_2/1 auto off > > $ rdma statistic qp bind link mlx5_2/1 lqpn 178 > > $ rdma statistic qp unbind link mlx5_2/1 cntn 4 lqpn 178 > > $ rdma statistic show > > $ rdma statistic qp mode > > > > Thanks > > > > Mark Zhang (16): > > net/mlx5: Add rts2rts_qp_counters_set_id field in hca cap > > RDMA/restrack: Introduce statistic counter > > RDMA/restrack: Add an API to attach a task to a resource > > RDMA/restrack: Make is_visible_in_pid_ns() as an API > > RDMA/counter: Add set/clear per-port auto mode support > > RDMA/counter: Add "auto" configuration mode support > > IB/mlx5: Support set qp counter > > IB/mlx5: Add counter set id as a parameter for mlx5_ib_query_q_counters() > > IB/mlx5: Support statistic q counter configuration > > RDMA/nldev: Allow counter auto mode configuration through RDMA netlink > > RDMA/netlink: Implement counter dumpit callback > > IB/mlx5: Add counter_alloc_stats() and counter_update_stats() support > > RDMA/core: Get sum value of all counters when perform a sysfs stat read > > RDMA/counter: Allow manual mode configuration support > > RDMA/nldev: Allow counter manual mode configuration through RDMA netlink > > RDMA/nldev: Allow get counter mode through RDMA netlink > > > > drivers/infiniband/core/Makefile | 2 +- > > drivers/infiniband/core/counters.c | 652 +++++++++++++++++++++++++++ > > drivers/infiniband/core/device.c | 14 + > > drivers/infiniband/core/nldev.c | 427 +++++++++++++++++- > > drivers/infiniband/core/restrack.c | 49 +- > > drivers/infiniband/core/restrack.h | 3 + > > drivers/infiniband/core/sysfs.c | 10 +- > > drivers/infiniband/core/verbs.c | 9 + > > drivers/infiniband/hw/mlx5/main.c | 88 +++- > > drivers/infiniband/hw/mlx5/mlx5_ib.h | 6 + > > drivers/infiniband/hw/mlx5/qp.c | 76 +++- > > include/linux/mlx5/mlx5_ifc.h | 4 +- > > include/linux/mlx5/qp.h | 1 + > > include/rdma/ib_verbs.h | 32 ++ > > include/rdma/rdma_counter.h | 64 +++ > > include/rdma/restrack.h | 4 + > > include/uapi/rdma/rdma_netlink.h | 52 ++- > > 17 files changed, 1462 insertions(+), 31 deletions(-) > > create mode 100644 drivers/infiniband/core/counters.c > > create mode 100644 include/rdma/rdma_counter.h > > > > -- > > 2.20.1 > > >
Attachment:
signature.asc
Description: PGP signature
