Hi Trond, Anna, We currently have several field installations containing NFS and SunRPC-related patches that greatly improve performance of NFSv3 clients over RDMA setups, where link aggregation is not supported. I would like work to integrate several of these changes to upstream, and discuss their implementation. We managed to get a bandwidth of 33 GB/sec from single node NFSv3 mount, and later around 92 GB/sec from a single mount using further enhancements in RPC request dispatch. The main change allows specifying multiple target IP addresses in a single mount, that combined with nconnect and multiple floating IPs, provides load balancing over several target nodes. This is good for systems where load balancing is managed by moving a group of floating IP addresses. This works especially well on RoCE setups. The networking setup on these clients comprises of multiple RDMA network interfaces that are connected to the same network, and each has its own IP address. The proposed change specifies a new `remoteports=<IP-addresses-ranges>` mount option providing a group of IP addresses, from which `nconnect` at sunrpc scope picks target transport address in round-robin. There's also an accompanying `localports` parameter that allows local address bind so that the source port is better controlled, in a way to ensure that transports are not hogging a single local interface. So essentially, this is a form of session trunking, that can be thought as an extension to the existing `nconnect` parameter. To my understanding NFSv4.x with pNFS has advanced dynamic transport management logic along file layouts supporting stripe over file offsets, however there are cases in which we would like to achieve good performance even with the older protocol. Before I adjust the patches I'm testing for v5.11, do you see other implementation or user interface considerations I should take into account? Thanks -- Dan Aloni
