On 11.02.2020 20:58, Chuck Lever wrote:
The @nents value that was passed to ib_dma_map_sg() has to be passed
to the matching ib_dma_unmap_sg() call. If ib_dma_map_sg() choses to
concatenate sg entries, it will return a different nents value than
it was passed.
The bug was exposed by recent changes to the AMD IOMMU driver.
This seems to fail differently on my system; mount fails with:
mount.nfs: mount system call failed
and the kernel log reports:
[ 38.890344] NFS: Registering the id_resolver key type
[ 38.890351] Key type id_resolver registered
[ 38.890352] Key type id_legacy registered
[ 38.901799] NFS: nfs4_discover_server_trunking unhandled error -5.
Exiting with error EIO
[ 38.901817] NFS4: Couldn't follow remote path
amd_iommu=off still works
One detail I accidentally left out of the original report is that the
server (intel system) is running Ubuntu 20.04 ("beta") userspace, and
AMD clients are Ubuntu 19.10 userspace. Although I dont believe this to
matter at this point.
Reported-by: Andre Tomt <andre@xxxxxxxx>
Suggested-by: Robin Murphy <robin.murphy@xxxxxxx>
Fixes: 1f541895dae9 ("xprtrdma: Don't defer MR recovery if ro_map fails")
Signed-off-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
---
net/sunrpc/xprtrdma/frwr_ops.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
Hey Andre, please try this out. It just reverts the bit of brokenness that
Robin observed this morning. I've done basic testing here with Intel
IOMMU systems, no change in behavior (ie, all good to go).
diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c
index 095be887753e..449bb51e4fe8 100644
--- a/net/sunrpc/xprtrdma/frwr_ops.c
+++ b/net/sunrpc/xprtrdma/frwr_ops.c
@@ -313,10 +313,9 @@ struct rpcrdma_mr_seg *frwr_map(struct rpcrdma_xprt *r_xprt,
break;
}
mr->mr_dir = rpcrdma_data_dir(writing);
+ mr->mr_nents = i;
- mr->mr_nents =
- ib_dma_map_sg(ia->ri_id->device, mr->mr_sg, i, mr->mr_dir);
- if (!mr->mr_nents)
+ if (!ib_dma_map_sg(ia->ri_id->device, mr->mr_sg, i, mr->mr_dir))
goto out_dmamap_err;
ibmr = mr->frwr.fr_mr;
