If the subject is changed to "IB/rxe: IB_WR_REG_MR does not capture MR's
iova field", it is better.
Zhu Yanjun
On 2018/11/26 6:13, Chuck Lever wrote:
FRWR memory registration is done with a series of calls and WRs.
1. ULP invokes ib_dma_map_sg()
2. ULP invokes ib_map_mr_sg()
3. ULP posts an IB_WR_REG_MR on the Send queue
Step 2 generates an iova. It is permissible for ULPs to change
this iova (with certain restrictions) between steps 2 and 3.
rxe_map_mr_sg captures the MR's iova but later when rxe processes
the REG_MR WR, it ignores the MR's iova field. If a ULP alters the
MR's iova after step 2 but before step 3, rxe never captures that
change.
When the remote sends an RDMA Read targeting that MR, rxe looks up
the R_key, but the altered iova does not match the iova stored in
the MR, causing the RDMA Read request to fail.
Reported-by: Anna Schumaker <schumaker.anna@xxxxxxxxx>
Signed-off-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
---
Hi-
This seems to address the problem Anna found:
https://marc.info/?l=linux-rdma&m=154273696218072&w=2
I imagine there is more to it than this, but NFS/RDMA seems to work
on rxe again with the XID-in-offset patch and this fix applied.
drivers/infiniband/sw/rxe/rxe_req.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
index 6c361d70d7cd..46f62f71cd28 100644
--- a/drivers/infiniband/sw/rxe/rxe_req.c
+++ b/drivers/infiniband/sw/rxe/rxe_req.c
@@ -643,6 +643,7 @@ int rxe_requester(void *arg)
rmr->access = wqe->wr.wr.reg.access;
rmr->lkey = wqe->wr.wr.reg.key;
rmr->rkey = wqe->wr.wr.reg.key;
+ rmr->iova = wqe->wr.wr.reg.mr->iova;
wqe->state = wqe_state_done;
wqe->status = IB_WC_SUCCESS;
} else {
