On Wed, Jun 05, 2019 at 11:25:49AM +0300, Leon Romanovsky wrote:
> On Wed, Jun 05, 2019 at 10:21:26AM +0300, Shamir Rabinovitch wrote:
> > On Thu, May 30, 2019 at 03:24:05PM +0300, Shamir Rabinovitch wrote:
> > > This patch set complete the cleanup done in the driver/verbs/uverbs
> > > where the dependency of the code in the ib_x uobject pointer was
> > > removed.
> > >
> > > The uobject pointer is removed from the ib_pd as last step
> > > before I can start adding the pd sharing code.
> > >
> > > The rdma/netlink code now don't have dependency in the ib_pd
> > > uobject pointer and can report multiple context id that point
> > > to same ib_pd.
> > >
> > > Using iproute2 I can test the modified rdma/netlink:
> > > [root@qemu-fc29 iproute2]# rdma/rdma res show pd dev mlx4_0
> > > dev mlx4_0 pdn 0 local_dma_lkey 0x8000 users 4 comm [ib_core]
> > > dev mlx4_0 pdn 1 local_dma_lkey 0x8000 users 4 comm [ib_core]
> > > dev mlx4_0 pdn 2 local_dma_lkey 0x8000 users 5 comm [ib_ipoib]
> > > dev mlx4_0 pdn 3 local_dma_lkey 0x8000 users 5 comm [ib_ipoib]
> > > dev mlx4_0 pdn 4 local_dma_lkey 0x8000 users 0 comm [ib_srp]
> > > dev mlx4_0 pdn 5 local_dma_lkey 0x8000 users 0 comm [ib_srpt]
> > > dev mlx4_0 pdn 6 local_dma_lkey 0x0 users 2 ctxn 0 pid 7693 comm ib_send_bw
> > > dev mlx4_0 pdn 7 local_dma_lkey 0x0 users 2 ctxn 1 pid 7694 comm ib_send_bw
> > >
> > > Changelog:
> > >
> > > v1->v2
> > > * 1 patch from v1 applied (Jason)
> > > * Fix uobj_get_obj_read macro (Jason)
> > > * Do not allocate memory when fixing uobj_get_obj_read (Jason)
> > > * Fix uobj_get_obj_read macro (Jason)
> > > * rdma/netlink can now work as before (Leon)
> > >
> > > v2->v3:
> > > * rdma/netlink nest multiple context ids of same ib_pd (Leon)
> > >
> > > Shamir Rabinovitch (4):
> > > RDMA/uverbs: uobj_get_obj_read should return the ib_uobject
> > > RDMA/uverbs: uobj_put_obj_read macro should be removed
> > > RDMA/nldev: ib_pd can be pointed by multiple ib_ucontext
> > > IB/{core,hw}: ib_pd should not have ib_uobject pointer
> > >
> > > drivers/infiniband/core/nldev.c | 129 +++++++++++-
> > > drivers/infiniband/core/uverbs_cmd.c | 218 +++++++++++++--------
> > > drivers/infiniband/core/verbs.c | 1 -
> > > drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 1 -
> > > drivers/infiniband/hw/mlx5/main.c | 1 -
> > > drivers/infiniband/hw/mthca/mthca_qp.c | 3 +-
> > > include/rdma/ib_verbs.h | 1 -
> > > include/rdma/uverbs_std_types.h | 11 +-
> > > include/uapi/rdma/rdma_netlink.h | 3 +
> > > 9 files changed, 273 insertions(+), 95 deletions(-)
> > >
> > > --
> > > 2.20.1
> > >
> >
> > Jason, Leon, can you please review this patch set ?
>
> I'm sorry for the delay.
>
> >
> > Anything missing from my side here?
>
> Can you please post rdmatool output for shared PD?
> In such case, all those shared PD need to have same PDN.
Leon I do not have all the pieces in place for this yes.
I only tested that current rdmatool can cope with netlink message that
has nested context ids as it will happen in shared pd case.
I'll do that once we have the export_to_fd & import_pd & import_mr verbs
in place.
>
> Thanks
