On 12.02.2020 14:48, Chuck Lever wrote:
Verified working (with the patch correction) in my environment, with some quick testing (mount + some random and bulk I/O)On Feb 12, 2020, at 8:43 AM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: The @nents value that was passed to ib_dma_map_sg() has to be passed to the matching ib_dma_unmap_sg() call. If ib_dma_map_sg() choses to concatenate sg entries, it will return a different nents value than it was passed. The bug was exposed by recent changes to the AMD IOMMU driver, which enabled sg entry concatenation. Looking all the way back to 4143f34e01e9 ("xprtrdma: Port to new memory registration API") and reviewing other kernel ULPs, it's not clear that the frwr_map() logic was ever correct for this case. Reported-by: Andre Tomt <andre@xxxxxxxx> Suggested-by: Robin Murphy <robin.murphy@xxxxxxx> Signed-off-by: Chuck Lever <chuck.lever@xxxxxxxxxx> --- include/trace/events/rpcrdma.h | 6 ++++-- net/sunrpc/xprtrdma/frwr_ops.c | 13 +++++++------ 2 files changed, 11 insertions(+), 8 deletions(-) Hi Andre, here's take 2, based on the trace data you sent me. Please let me know if this one fares any better. Changes since v1: - Ensure the correct nents value is passed to ib_map_mr_sg - Record the mr_nents value in the MR trace points
client, 5.5.3 + patch + amd iommu on = OK client, 5.5.3 + patch + amd iommu off = OK client, 5.6-rc1 + patch + amd iommu on = OK server, 5.5.3 + patch + intel iommu on = OK Thanks!
