On Tue, Jan 22, 2019 at 12:51:38PM -0700, Jason Gunthorpe wrote:
> On Tue, Jan 22, 2019 at 09:39:42PM +0200, Leon Romanovsky wrote:
> > On Tue, Jan 22, 2019 at 12:09:49PM -0700, Jason Gunthorpe wrote:
> > > On Tue, Jan 22, 2019 at 08:18:56PM +0200, Leon Romanovsky wrote:
> > > > From: Leon Romanovsky <leonro@xxxxxxxxxxxx>
> > > >
> > > > PD, MR and QP objects have parents objects: contexts and PDs.
> > > > The exposed parent IDs allow to correlate various objects
> > > > and simplify debug investigation.
> > > >
> > > > Signed-off-by: Leon Romanovsky <leonro@xxxxxxxxxxxx>
> > > > drivers/infiniband/core/nldev.c | 41 ++++++++++++++++++++++++++++----
> > > > include/uapi/rdma/rdma_netlink.h | 1 +
> > > > 2 files changed, 38 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
> > > > index 3b24838fe6e0..9bcfd3e4e099 100644
> > > > +++ b/drivers/infiniband/core/nldev.c
> > > > @@ -111,6 +111,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> > > > [RDMA_NLDEV_ATTR_RES_CQN] = { .type = NLA_U32 },
> > > > [RDMA_NLDEV_ATTR_RES_MRN] = { .type = NLA_U32 },
> > > > [RDMA_NLDEV_ATTR_RES_CM_IDN] = { .type = NLA_U32 },
> > > > + [RDMA_NLDEV_ATTR_RES_CTXN] = { .type = NLA_U32 },
> > > > };
> > > >
> > > > static int put_driver_name_print_type(struct sk_buff *msg, const char *name,
> > > > @@ -375,25 +376,54 @@ static bool fill_res_entry(struct ib_device *dev, struct sk_buff *msg,
> > > >
> > > > static bool fill_res_ids(struct sk_buff *msg, struct rdma_restrack_entry *res)
> > > > {
> > > > - bool ret;
> > > > + bool ret = false;
> > > >
> > > > switch (res->type) {
> > > > - case RDMA_RESTRACK_PD:
> > > > + case RDMA_RESTRACK_PD: {
> > > > + struct ib_pd *pd = container_of(res, struct ib_pd, res);
> > > > +
> > > > ret = nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_PDN,
> > > > rdma_res_to_id(res));
> > > > + if (!ret && !rdma_is_kernel_res(res))
> > > > + ret = nla_put_u32(
> > > > + msg, RDMA_NLDEV_ATTR_RES_CTXN,
> > > > + rdma_res_to_id(&pd->uobject->context->res));
> > >
> > > uobject->context cannot be accessed without proper uobject locking.
> >
> > How can context "disappear" if we are holding PD which is part of that context?
>
> You want to rely on the restrack HW object locking to protect the
> uobject? Sort of backwards.. Today the uobject exists strictly longer
> the HW object, but I'm not sure if that is a great assumption to bake
> in here..
uverbs PD doesn't exist without ucontext, it is verbs requirement.
>
> > > See the recent email to Shamir.
> > >
> > > Shamir is also working to eliminate pd->uobject so we should not have
> > > more references being added.
> >
> > Was anything accepted so I can use instead? If not, I will update the
> > code, once it will be part of rdma-next.
>
> pd's won't have a single ucontext as parent. Do not add new user ABI
> that assume this.
I don't assume that and once PD will support multiple contexts, the
author will add two new netlink objects .._TABLE and ..._ENTRY and
will fill RDMA_NLDEV_ATTR_RES_CTXN under nesting.
The user space code checks availability of RDMA_NLDEV_ATTR_RES_CTXN
and nesting handles the flows accordingly.
As an example, this change to rdmatool, to try to read
RDMA_NLDEV_ATTR_RES_CTXN outside of nesting:
diff --git a/rdma/res-pd.c b/rdma/res-pd.c
index 07c836e8..928b75d2 100644
--- a/rdma/res-pd.c
+++ b/rdma/res-pd.c
@@ -117,6 +117,12 @@ int res_pd_parse_cb(const struct nlmsghdr *nlh, void *data)
!tb[RDMA_NLDEV_ATTR_RES_PD])
return MNL_CB_ERROR;
+ if (tb[RDMA_NLDEV_ATTR_RES_CTXN])
+ pr_out("ctxn = %u\n",
+ mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_RES_CTXN]));
+ else
+ pr_out("not found\n");
+
name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
nla_table = tb[RDMA_NLDEV_ATTR_RES_PD];
[leonro@server ~]$ /images/leonro/src/iproute2/rdma/rdma res show pd
not found
dev mlx5_0 pdn 1 users 5 comm [ib_core]
dev mlx5_0 pdn 2 users 6 comm [ib_ipoib]
dev mlx5_0 pdn 3 users 0 comm [mlx5_ib]
dev mlx5_0 pdn 5 users 2 ctxn 1 pid 644 comm ibv_rc_pingpong
^^^^^ context exists
Thanks
>
> Jason
Attachment:
signature.asc
Description: PGP signature
