On 9/7/2017 11:08 AM, Jason Gunthorpe wrote:
On Thu, Sep 07, 2017 at 09:17:16AM -0400, Tom Talpey wrote:
Why is waiting for the send completion so fundamentally different from
waiting for the remote RPC reply?
I would say that 99% of time the send completion and RPC reply
completion will occure approximately concurrently.
Absolutely not. The RPC reply requires upper layer processing at
the server, which involves work requests, context switches, file
I should have said '99% of the time the SEND will occure approximately
concurrently or sooner'
Ok, that I agree with. Unfortunately though, the code has to handle
either sequence, and sends do complete after replies in many cases.
One reason for that is the RNIC and network, but another is more
insidious. When sends and receives go to separate CQs, there's
basically no causality between the two, and they synchronize with
different MSI vectors (and therefore CPU cores), different locks,
and different upcall code paths. There's no way to sort that out
without serializing everything much too heavily.
Tom.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
