The RPC/RDMA version 1 protocol allows clients to provision more than one Write chunk in each RPC/RDMA message. The Linux NFS client never has need to construct a message with multiple Write chunks, so the Linux NFS server has never implemented support for them. At testing events, we discovered that the Solaris NFS client can emit such requests on occasion, but only when an application invokes readv(2) on a "forcedirectio" mount -- rare, indeed. Even so, it's been on my "to-do" list for quite some time to get the Linux NFS server to handle multiple Write chunks. While addressing the recent NFSD/RDMA bug with Linux filesystems that do not have a .read_splice method [1], I realized that it was time to get this one off my plate. So here is an attempt to support NFS/RDMA clients that send multiple Write chunks. To do this generically requires more xdr_buf slicing and dicing than the simple "zero or one" implementation. At the same time, the ability to send RDMA Write requests _outside_ the .xpo_sendto path is introduced. Extensive testing has not revealed any functional or performance regression with this change. Thoughts and comments are welcome. [1] - https://bugzilla.kernel.org/show_bug.cgi?id=198053 --- Chuck Lever (11): SUNRPC: Adjust synopsis of xdr_buf_subsegment() svcrdma: Clean up RDMA Write path NFSD: Invoke svc_encode_read_payload in "read" NFSD encoders svcrdma: Post RDMA Writes while XDR encoding replies svcrdma: Clean up svc_rdma_encode_reply_chunk() svcrdma: Cache number of Write chunks svcrdma: Add a data structure to track READ payloads svcrdma: Add svc_rdma_skip_payloads() svcrdma: Support multiple READ payloads when pulling up svcrdma: Support multiple READ payloads in svc_rdma_map_reply_msg() svcrdma: Support multiple Write chunks in svc_rdma_send_reply_chunk fs/nfsd/nfs3xdr.c | 4 + fs/nfsd/nfs4xdr.c | 3 + fs/nfsd/nfsxdr.c | 4 + include/linux/sunrpc/svc_rdma.h | 24 +- include/trace/events/rpcrdma.h | 16 +- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 32 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 134 +++++--- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 556 ++++++++++++++++++++------------ 8 files changed, 486 insertions(+), 287 deletions(-) -- Chuck Lever
