On Tue, May 21, 2019 at 05:32:30AM -0700, Nirranjan Kirubaharan wrote:
> In iw_cxgb4, Added wait in destroy_qp() so that all references to
> qp are dereferenced and qp is freed in destroy_qp() itself.
> This ensures freeing of all QPs before invocation of
> dealloc_ucontext(), which prevents loss of in use qpids stored
> in ucontext.
>
> Signed-off-by: Nirranjan Kirubaharan <nirranjan@xxxxxxxxxxx>
> Reviewed-by: Potnuri Bharat Teja <bharat@xxxxxxxxxxx>
> v2:
> - Used kref instead of qid count.
> v3:
> - Ensured freeing of qp in destroy_qp() itself.
> drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 1 +
> drivers/infiniband/hw/cxgb4/qp.c | 7 ++++++-
> 2 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
> index 916ef982172e..10c3e5e9d3de 100644
> +++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
> @@ -497,6 +497,7 @@ struct c4iw_qp {
> struct work_struct free_work;
> struct c4iw_ucontext *ucontext;
> struct c4iw_wr_wait *wr_waitp;
> + struct completion qp_rel_comp;
> };
>
> static inline struct c4iw_qp *to_c4iw_qp(struct ib_qp *ibqp)
> diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c
> index e92b9544357a..ea0b7014fb03 100644
> +++ b/drivers/infiniband/hw/cxgb4/qp.c
> @@ -905,7 +905,7 @@ static void free_qp_work(struct work_struct *work)
> ucontext ? &ucontext->uctx : &rhp->rdev.uctx, !qhp->srq);
>
> c4iw_put_wr_wait(qhp->wr_waitp);
> - kfree(qhp);
> + complete(&qhp->qp_rel_comp);
> }
>
> static void queue_qp_free(struct kref *kref)
> @@ -2120,7 +2120,11 @@ int c4iw_destroy_qp(struct ib_qp *ib_qp, struct ib_udata *udata)
>
> c4iw_qp_rem_ref(ib_qp);
>
> + wait_for_completion(&qhp->qp_rel_comp);
> +
> pr_debug("ib_qp %p qpid 0x%0x\n", ib_qp, qhp->wq.sq.qid);
> +
> + kfree(qhp);
> return 0;
> }
>
> @@ -2184,6 +2188,7 @@ struct ib_qp *c4iw_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *attrs,
> (sqsize + rhp->rdev.hw_queue.t4_eq_status_entries) *
> sizeof(*qhp->wq.sq.queue) + 16 * sizeof(__be64);
> qhp->wq.sq.flush_cidx = -1;
> + init_completion(&qhp->qp_rel_comp);
> if (!attrs->srq) {
> qhp->wq.rq.size = rqsize;
> qhp->wq.rq.memsize =
So now you don't need the work queue at all, and you are back to using
the to_c4iw_qp(qp)->kref as not-a-kref.
Use the normal pattern please.
Change c3iw_qp_rem_ref to use a refcount not kref and trigger
complete() when the refcount goes t 0.
Move all of queue_qp_free into ciw_destroy_qp
Remove the work item entirely.
Jason
