On 05/01/2022 00:40, Tom Talpey wrote:
>
> On 12/28/2021 3:07 AM, Li Zhijian wrote:
>
>> +static enum resp_states process_flush(struct rxe_qp *qp,
>> + struct rxe_pkt_info *pkt)
>> +{
>> + u64 length = 0, start = qp->resp.va;
>> + u32 sel = feth_sel(pkt);
>> + u32 plt = feth_plt(pkt);
>> + struct rxe_mr *mr = qp->resp.mr;
>> +
>> + if (sel == IB_EXT_SEL_MR_RANGE)
>> + length = qp->resp.length;
>> + else if (sel == IB_EXT_SEL_MR_WHOLE)
>> + length = mr->cur_map_set->length;
>
> I noticed another issue in this. When the "Memory Region" flag is
> set, the RETH:VA field in the request is ignored, in addition to
> the RETH:DMALEN. Therefore, both the start and length must be set
> from the map.
Yes, that's true
Actually, i have drafted a new version last week when you first time pointed out this.
https://github.com/zhijianli88/linux/commit/a368d821b163f74836d527638625d414c7a62d00#diff-1b675c68d2f5f664abefea05cbf415f6307c9920b31a13bed41105c4a84e0259R641
the latest code is like:
if (sel == IB_EXT_SEL_MR_RANGE) {
start = qp->resp.va;
length = qp->resp.length;
} else { /* sel == IB_EXT_SEL_MR_WHOLE */
start = mr->cur_map_set->iova;
length = mr->cur_map_set->length;
}
>
> Is the mr->cur_map_set[0]->addr field a virtual address, or a
> physical?
mr->cur_map_set[0]->addr is a virtual address in kernel space.
nvdimm_flush_iova() will accept a user space address(alloc(), mmap() by applications),
this address could be transited to a kernel virtual address by lookup_iova() where it
will refer to mr->cur_map_set[n]->addr.
So i think we should use qp->resp.va and mr->cur_map_set->iova that are both user
space address.
> I can't find the cur_map_set in any patch.
cur_map_set will be filled in rxe_mr_init_user(),
rxe_mr_init_user()
-> ib_umem_get()
-> pin_user_pages_fast()
-> update cur_map_set
BTW: some of you all comments are already addressed in
https://github.com/zhijianli88/linux/tree/rdma-flush
and it may forced update irregularly.
Thanks
Zhijian
> The virtual
> (mapped) address will be needed, to pass to arch_wb_cache_pmem().
>
> Something like this?
>
> if (sel == IB_EXT_SEL_MR_WHOLE) {
> length = mr->cur_map_set->length;
> start = mr->cur_map_set[0]->addr;
> }
>
> (in addition to my other review suggestions about 0-length, etc...)
>
> Tom.
