Hi Jason
on 2022/1/19 20:36, Jason Gunthorpe wrote:
On Wed, Jan 19, 2022 at 01:54:32AM +0000, lizhijian@xxxxxxxxxxx wrote:
On 18/01/2022 20:35, Jason Gunthorpe wrote:
On Tue, Jan 18, 2022 at 08:01:59AM +0000, yangx.jy@xxxxxxxxxxx wrote:
On 2022/1/17 21:16, Jason Gunthorpe wrote:
On Thu, Jan 13, 2022 at 11:03:50AM +0800, Xiao Yang wrote:
+static enum resp_states process_atomic_write(struct rxe_qp *qp,
+ struct rxe_pkt_info *pkt)
+{
+ struct rxe_mr *mr = qp->resp.mr;
+
+ u64 *src = payload_addr(pkt);
+
+ u64 *dst = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, sizeof(u64));
+ if (!dst || (uintptr_t)dst& 7)
+ return RESPST_ERR_MISALIGNED_ATOMIC;
It looks to me like iova_to_vaddr is completely broken, where is the
kmap on that flow?
Hi Jason,
I think rxe_mr_init_user() maps the user addr space to the kernel addr
space during memory region registration, the mapping records are saved
into mr->cur_map_set->map[x].
There is no way to touch user memory from the CPU in the kernel
That's absolutely right, but I don't think it references that user memory directly.
without calling one of the kmap's, so I don't know what this thinks it
is doing.
Jason
IMHO, for the rxe, rxe_mr_init_user() will call get_user_page() to pin iova first, and then
the page address will be recorded into mr->cur_map_set->map[x]. So that when we want
to reference iova's kernel address, we can call iova_to_vaddr() where it will retrieve its kernel
address by travel the mr->cur_map_set->map[x].
That flow needs a kmap
IIUC, this was a existing issue in iova_to_vaddr() right ?
Alternatively, can we have below changes to fix it simply?
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c
b/drivers/infiniband/sw/rxe/rxe_mr.c
index 0621d387ccba..978fdd23665c 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -260,7 +260,8 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start,
u64 length, u64 iova,
num_buf = 0;
}
- vaddr = page_address(sg_page_iter_page(&sg_iter));
+ // FIXME: don't forget to kunmap_local(vaddr)
+ vaddr =
kmap_local_page(sg_page_iter_page(&sg_iter));
if (!vaddr) {
pr_warn("%s: Unable to get virtual
address\n",
__func__);
Do you mean we should retrieve iova's page first, and the reference the kernel address by
kmap(), sorry for my stupid question ?
Going from struct page to something the kernel can can touch requires
kmap
Got it
Thanks
Zhijian
Jason
