> -----Original Message-----
> From: Chengguang Xu <cgxu519@xxxxxxxxxxxx>
> Sent: Monday, May 2, 2022 12:39 AM
> To: zyjzyj2000@xxxxxxxxx; jgg@xxxxxxxx; leon@xxxxxxxxxx
> Cc: linux-rdma@xxxxxxxxxxxxxxx; Chengguang Xu <cgxu519@xxxxxxxxxxxx>
> Subject: [RFC PATCH] RDMA/rxe: skip adjusting remote addr for write in retry operation
>
> For write request the remote addr will be sent only with first packet so we don't have to adjust wqe->iova in retry operation.
This is problematic for lossy networks. A very large read request, say 8MiB, sends 2048 packets in response without any acknowledgement
from the requester. If the packet loss rate was 1% the read request would never finish as the probability of sending 2048 packets without
loss is very small. The way the code works today is that the iova is adjusted, and if you are lucky the responder has already finished the
previous read operation and starts over with a new read reply starting with a first packet at iova. If you are less fortunate the previous
read reply has not finished and the responder will continue to work on it until it is finished before looking at the new read request wqe.
the completer will respond to each out of order packet by checking to see if it should start a retry but since it has already done so
it drops the packet. It's messy but one can make forward progress ~100 packets at a time. It would be faster if the responder saw that
the new request replaced the old one and stopped sending packets on the old read. I have no idea how CX NICs do this but just restarting
from scratch seems bad.
Bob
Signed-off-by: Chengguang Xu <cgxu519@xxxxxxxxxxxx>
---
drivers/infiniband/sw/rxe/rxe_req.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
index ae5fbc79dd5c..f08010651ef7 100644
--- a/drivers/infiniband/sw/rxe/rxe_req.c
+++ b/drivers/infiniband/sw/rxe/rxe_req.c
@@ -33,8 +33,6 @@ static inline void retry_first_write_send(struct rxe_qp *qp,
} else {
advance_dma_data(&wqe->dma, to_send);
}
- if (mask & WR_WRITE_MASK)
- wqe->iova += qp->mtu;
}
}
--
2.35.1
