Hi Bruce-
As promised, I'm resending the fix for 198053, now that the v5.6
merge window has closed. This fix gets splice-incapable file systems
working with NFS/RDMA. That's the first patch in this series.
We can discuss splitting the fix up again, if you so desire, but my
sense is that will make the fix more challenging to backport into
stable kernels.
The next logical step is to add support for multiple READ payloads
to the server's RPC-over-RDMA transport implementation. Subsequent
patches in this series start down that path. There is more work to
do to finish that task. Today I'm sending only what is code-complete
and working.
The primary issue is that today svcrdma assumes that rq_res's page
vector is exactly what needs to be pushed in a single Write chunk.
In other words, only one read payload is supported, and it has to
fit exactly into that page vector. And critically, the XDR pad for
that payload must not be included in the page vector.
I've already implemented changes to handle Writing more than one
chunk back to a client. See patches 4 and 7.
Patch 9 introduces a data structure to keep track of multiple Write
chunks and multiple read payloads. Next, the svc_rdma_sendto path
needs to be changed to use the information in this data structure to
exclude arbitrary segments of rq_res (ie, read payloads already sent
via explicit RDMA) when constructing each RPC/RDMA Reply.
Comments and input are welcome as always.
---
Chuck Lever (9):
nfsd: Fix NFSv4 READ on RDMA when using readv
NFSD: Clean up nfsd4_encode_readv
svcrdma: Avoid DMA mapping small RPC Replies
NFSD: Invoke svc_encode_read_payload in "read" NFSD encoders
svcrdma: Add trace point to examine client-provided write segment
svcrdma: De-duplicate code that locates Write and Reply chunks
svcrdma: Post RDMA Writes while XDR encoding replies
svcrdma: Refactor svc_rdma_sendto()
svcrdma: Add data structure to track READ payloads
fs/nfsd/nfs3xdr.c | 4
fs/nfsd/nfs4xdr.c | 32 ++--
fs/nfsd/nfsxdr.c | 4
include/linux/sunrpc/svc.h | 3
include/linux/sunrpc/svc_rdma.h | 21 ++
include/linux/sunrpc/svc_xprt.h | 2
include/trace/events/rpcrdma.h | 47 +++++
net/sunrpc/svc.c | 16 ++
net/sunrpc/svcsock.c | 8 +
net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 2
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 58 +++++--
net/sunrpc/xprtrdma/svc_rdma_rw.c | 42 +++--
net/sunrpc/xprtrdma/svc_rdma_sendto.c | 248 +++++++++++++---------------
net/sunrpc/xprtrdma/svc_rdma_transport.c | 1
14 files changed, 308 insertions(+), 180 deletions(-)
--
Chuck Lever
