On 7/14/2015 7:35 PM, Jason Gunthorpe wrote:
On Tue, Jul 14, 2015 at 07:12:01PM +0300, Sagi Grimberg wrote:
The ULP doesn't care if it needs to reserver the slot, and it generally
doesn't care about the notification either unless it needs to handle an
error.
That's generally correct. But the ULP may want to suppress notifications
of other posts as well (for example a storage initiator may want to
suppress send completions since it needs to reserve the request context
until it receives the status anyway - which is a receive completion). It
needs to take what was posted and what not into account to avoid
wrapping.
I don't see how this is a concern, API wise. All callers are paring
the MR post with a real op anyhow.
If I'm not mistaken iWARP requires to ask for a completion once every
send-queue length and empirically, IB drivers requires it too. So again,
I don't think implicit posting is a very good idea.
Yes, all proper use of RDMA requires counting SQEs consumed and
freeing them only on completion. Nobody should be using the EAGAIN
scheme. But it is trivial to handle, you don't start an op until you
can post the entire compound SQE, and you must always tag the last SQE
for signal if the # of SQEs left drops below the # required for the
largest compound (or another similar scheme).
But you should never have to signal on the MR post, it should always
be at the end of the compound, on the RDMA, SEND, or invalidate..
That is just basic 'how do you use a SQ properly', every ULP should
do it, wrappering posts in an API doesn't change anything fundamental.
Well, I still think keeping the post interface is better.
But, if people think that it's better to have an API that does implicit
posting always without notification, and then silently consume error or
flush completions. I can try and look at it as well.
Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
