On 04/26/2016 09:28 AM, Steve Wise wrote:
On 4/26/2016 11:04 AM, Bart Van Assche wrote:
On 04/26/2016 09:00 AM, Steve Wise wrote:
On 4/26/2016 10:55 AM, Chuck Lever wrote:
On Apr 26, 2016, at 11:43 AM, Christoph Hellwig <hch@xxxxxx> wrote:
Just curious: what takes care of draining SRQs when we unregister them
after all queues are gone?
Conversely, I would think that a consumer that uses SRQ
would want a similar drain mechanism to guarantee there
are no more posted Receives for the associated QP, and
thus it is safe to destroy it.
I don't think there is anything now that handles draining an SRQ, nor
ensuring a QP's RQEs are completed/consumed from its SRQ when draining
the QP. Sounds like work is needed here.
But are there any kernel SRQ consumers at this point?
I think the following two:
$ git grep -nHE '(ib|rdma)_create_srq' drivers/infiniband/ulp
drivers/infiniband/ulp/ipoib/ipoib_cm.c:1521: priv->cm.srq =
ib_create_srq(priv->pd, &srq_init_attr);
drivers/infiniband/ulp/srpt/ib_srpt.c:2723: sdev->srq =
ib_create_srq(sdev->pd, &srq_attr);
If we make the requirement that the ib_drain_rq() caller must consume
all completions for all QPs attached to an SRQ if they are outstanding,
then I think we can modify ib_drain_rq() to post the drain recv WR to
the SRQ. It should work, right?
At least the ib_srpt driver already guarantees that no further receive
completions will be generated before ib_destroy_qp() is called. But
posting an additional receive WR on the SRQ from inside ib_drain_rq()
shouldn't hurt.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
