On 2/10/2020 1:32 PM, Jason Gunthorpe wrote:
On Mon, Feb 10, 2020 at 12:36:02PM -0500, Dennis Dalessandro wrote:
On 2/10/2020 8:31 AM, Jason Gunthorpe wrote:
On Mon, Feb 10, 2020 at 08:18:05AM -0500, Dennis Dalessandro wrote:
This patch series is an accelerated ipoib using the rdma netdev mechanism
already present in ipoib. A new device capability bit,
IB_DEVICE_RDMA_NETDEV_OPA, triggers ipoib to create a datagram QP using the
IB_QP_CREATE_NETDEV_USE.
The highlights include:
- Sharing send and receive resources with VNIC
- Allows for switching between connected mode and datagram mode
There is still value in connected mode?
It's really a compatibility thing. If someone wants to change modes that
will work. There won't be any benefit to connected mode though. The goal is
just to not break.
I am a bit confused by this.. I thought the mlx5 implementation
already could select connected mode?
Why were core ipoib changes needed?
I don't think so, patch 15/16 seemed to be necessary to get connected
mode to work with the rdma netdev.
The patches are fully bisectable and stepwise implement the capability.
This is alot of code to send without a performance
justification.. What is it? Is it worth while?
It avoids the scalability problem of connected mode, the number of QPs.
Incoming packets are spread into multiple receive contexts increasing
parallelism. The MTU is increased to allows 10K. It also reduces/removes the
verbs TX overhead by allowing packets to be sent through the SDMA engines
directly.
No numbers to share?
No numbers directly but I can say that AIP enables line-rate performance
between two nodes with Datagram Mode, it also provides IPoFabric latency
improvements relative to standard Datagram Mode without AIP.
-Denny
