On 7/14/2015 6:33 PM, Christoph Hellwig wrote:
On Tue, Jul 14, 2015 at 11:39:24AM +0300, Sagi Grimberg wrote:
This is exactly what I don't want to do. I don't think that implicit
posting is a good idea for reasons that I mentioned earlier:
"This is where I have a problem. Providing an API that may or may not
post a work request on my QP is confusing, and I don't understand its
semantics at all. Do I need to reserve slots on my QP? should I ask for
a completion? If we suppress the completion will I see an error
completion? What should I expect to find in the wr_id?"
We're much better off with keeping the post interface in place but
have it much simpler.
The ULP doesn't care if it needs to reserver the slot, and it generally
doesn't care about the notification either unless it needs to handle an
error.
That's generally correct. But the ULP may want to suppress notifications
of other posts as well (for example a storage initiator may want to
suppress send completions since it needs to reserve the request context
until it receives the status anyway - which is a receive completion). It
needs to take what was posted and what not into account to avoid
wrapping.
If I'm not mistaken iWARP requires to ask for a completion once every
send-queue length and empirically, IB drivers requires it too. So again,
I don't think implicit posting is a very good idea.
The completions are another mad mightmare in the RDMA stack API. Every
other subsystem would just allow submitter to attach a function pointer
that handles the completion to the posted item.
This is actually a good idea. And it can be easily added I think (at
least to mlx drivers which I better). It can help removing a switch
statement for the ULPs when handling the completions.
But no, the RDMA stack
instead require ID allocators and gigantic boilerplate code in the
consumer to untangle that gigantic mess again.
I don't think the wr_id was ever intended to be an allocated tag. It's
the ULP context, usually a pointer to a transaction structure. Even
with passing a function pointer you need to get back your original
context don't you?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
