On 6/12/2017 12:55 PM, Jason Gunthorpe wrote:
On Mon, Jun 12, 2017 at 04:53:28PM +0000, Parav Pandit wrote:
From: Jason Gunthorpe [mailto:jgunthorpe@xxxxxxxxxxxxxxxxxxxx]
Sent: Monday, June 12, 2017 11:44 AM
To: Parav Pandit <parav@xxxxxxxxxxxx>
Cc: Bart Van Assche <Bart.VanAssche@xxxxxxxxxxx>; leon@xxxxxxxxxx;
dledford@xxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx; Idan Burstein
<idanb@xxxxxxxxxxxx>
Subject: Re: [PATCH rdma-next 0/3] Support out of order data placement
On Mon, Jun 12, 2017 at 04:42:27PM +0000, Parav Pandit wrote:
I would suggest at least using the inverted sense like Bart
describes in the kernel - every kernel ULP is safe.
I don't see a need to use inverted sense in code. I can surely make
documentation more descriptive as Bart suggested.
If this is 'better' then it should be on as much as possible, and I certianly
don't want to see kernel ULPs query caps and other pointless things when
they already, necessarily, deal with out of order.
Sure. Kernel ULPs and any other user ULPs can skip query caps.
No, they can't because only mlx5 accepts the new flag.
This is why inverting the flag in the kernel makes much more sense.
Absolutely. All iWARP adapters support multiple RDMA Writes in flight,
and their placement is not ordered. It's a quirk of the MLX devices,
and in turn their implementation of the IB protocol, that they place
writes in-order. I'm happy to see Mellanox become aware of this, it
limits write performance on networks with any significant latencies.
I agree with Jason, the bit should be 1 by default, if defined as you
propose. Out-of-order is the norm, not the exception, for ULPs.
Honestly, I think you should perhaps consider making it the default
on your devices, and allowing only MLX-aware ULPs to turn it off.
The Windows NDK API has a similar concept, by the way, it flips your
meaning and defaults to 0. Also, it's not settable by the ULP. The
flag isn't used for much, as a result.
NDK_ADAPTER_FLAG_IN_ORDER_DMA_SUPPORTED
https://msdn.microsoft.com/en-us/library/windows/hardware/hh439851(v=vs.85).aspx
Tom.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
