Hi Chuck, On 2024/10/17 9:24 PM, Chuck Lever wrote: > On Thu, Oct 17, 2024 at 10:52:19AM +0800, Kinglong Mee wrote: >> Hi Chuck, >> >> On 2024/10/16 9:57 PM, Chuck Lever wrote: >>> On Wed, Oct 16, 2024 at 07:48:25PM +0800, Kinglong Mee wrote: >>>> NFSd's DRC key contains peer port, but rpcrdma does't support port resue now. >>>> This patch supports resvport and resueport as tcp/udp for rpcrdma. >>> >>> An RDMA consumer is not in control of the "source port" in an RDMA >>> connection, thus the port number is meaningless. This is why the >>> Linux NFS client does not already support setting the source port on >>> RDMA mounts, and why NFSD sets the source port value to zero on >>> incoming connections; the DRC then always sees a zero port value in >>> its lookup key tuple. >>> >>> See net/sunrpc/xprtrdma/svc_rdma_transport.c :: handle_connect_req() : >>> >>> 259 /* The remote port is arbitrary and not under the control of the >>> 260 * client ULP. Set it to a fixed value so that the DRC continues >>> 261 * to be effective after a reconnect. >>> 262 */ >>> 263 rpc_set_port((struct sockaddr *)&newxprt->sc_xprt.xpt_remote, 0); >>> >>> >>> As a general comment, your patch descriptions need to explain /why/ >>> each change is being made. For example, the initial patches in this >>> series, although they properly split the changes into clean easily >>> digestible hunks, are somewhat baffling until the reader gets to >>> this one. This patch jumps right to announcing a solution. >> >> Thanks for your comment. >> >>> >>> There's no cover letter tying these changes together with a problem >>> statement. What problematic behavior did you see that motivated this >>> approach? >> >> We have a private nfs server, it's DRC checks the src port, but rpcrdma doesnot >> support resueport now, so we try to add it as tcp/udp. > > Thank you for clarifying! > > It's common for a DRC to key on the source port. Unfortunately, > IIRC, the Linux RDMA Connection Manager does not provide an API for > an RDMA consumer (such as the Linux NFS client) to set an arbitrary > source port value on the active side. rdma_bind_addr() works on the > listen side only. rdma_bind_addr() also works on client before rdma_resolve_addr.
