On 9/6/2017 3:39 PM, Jason Gunthorpe wrote:
On Wed, Sep 06, 2017 at 02:33:50PM -0400, Chuck Lever wrote:
B. Force RPC completion to wait for Send completion, which
would allow the post-v4.6 scatter-gather code to work
safely. This would need some guarantee that Sends will
always complete in a short period.
Why is waiting for the send completion so fundamentally different from
waiting for the remote RPC reply?
I would say that 99% of time the send completion and RPC reply
completion will occure approximately concurrently.
Absolutely not. The RPC reply requires upper layer processing at
the server, which involves work requests, context switches, file
system and disk i/o operations, and plain old queuing. RPC replies
typically can be expected in milliseconds from traditional HDDs,
and hundreds of microseconds from SSDs. Storage class memory is
reducing these by two orders of magnitude, but it's still not in
the range of a network round trip.
eg It is quite likely the RPC reply SEND carries an embeded ack
for the requesting SEND..
That's not what we've seen, even on InfiniBand. On adapters such as
iWARP, I will note, the send completion is often generated when
the adapter buffers the outgoing message, and has almost nothing
to do with the remote transport ack.
Tom.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
