On 09/11/2023 15:16, Greg Sword wrote: > On Thu, Nov 9, 2023 at 10:25 AM Zhijian Li (Fujitsu) > <lizhijian@xxxxxxxxxxx> wrote: >> >> >> >> On 06/11/2023 21:58, Zhu Yanjun wrote: >>> 在 2023/11/6 12:07, Zhijian Li (Fujitsu) 写道: >>>> >>>> >>>> On 03/11/2023 21:00, Zhu Yanjun wrote: >>>>> 在 2023/11/3 17:55, Li Zhijian 写道: >>>>>> I don't collect the Reviewed-by to the patch1-2 this time, since i >>>>>> think we can make it better. >>>>>> >>>>>> Patch1-2: Fix kernel panic[1] and benifit to make srp work again. >>>>>> Almost nothing change from V1. >>>>>> Patch3-5: cleanups # newly add >>>>>> Patch6: make RXE support PAGE_SIZE aligned mr # newly add, but not fully tested >>>>>> >>>>>> My bad arm64 mechine offten hangs when doing blktests even though i use the >>>>>> default siw driver. >>>>>> >>>>>> - nvme and ULPs(rtrs, iser) always registers 4K mr still don't supported yet. >>>>> >>>>> Zhijian >>>>> >>>>> Please read carefully the whole discussion about this problem. You will find a lot of valuable suggestions, especially suggestions from Jason. >>>> >>>> Okay, i will read it again. If you can tell me which thread, that would be better. >>>> >>>> >>>>> >>>>> From the whole discussion, it seems that the root cause is very clear. >>>>> We need to fix this prolem. Please do not send this kind of commits again. >>>>> >>>> >>>> Let's think about what's our goal first. >>>> >>>> - 1) Fix the panic[1] and only support PAGE_SIZE MR >>>> - 2) support PAGE_SIZE aligned MR >>>> - 3) support any page_size MR. >>>> >>>> I'm sorry i'm not familiar with the linux MM subsystem. It seem it's safe/correct to access >>>> address/memory across pages start from the return of kmap_loca_page(page). >>>> In other words, 2) is already native supported, right? >>> >>> Yes. Please read the comments from Jason, Leon and Bart. They shared a lot of good advice. >> >> I read the whole discussion again, but I believed i still missed a lost. >> >> >>> From them, we can know the root cause and how to fix this problem. >> >> I don't think i misunderstood the root cause: >> RXE splits memory into PAGE_SIZE units in the xarray. As a result, when we extract an address from the xarray, >> we should not access address beyond a PAGE_SIZE window. >> >> IIUC, then how to fix it? >> - I'm not going to "removing page_size set", it's out of this patch scope. >> Feel free to do the cleanup separately. >> - I'm not going to fix the NVMe/rtrs etc problems in this patch set when 64K page is enabled. >> But RXE will tell its callers explicitly "RXE don't don't support such page_size" >> - I didn't state RXE supports PAGE_SIZE aligned page_size MR before refactoring rxe_map_mr_sg(), >> because I worry about it was not correct to access address beyond the PAGE_SIZE window. >> >> What I should do next? >> Just state "RXE support PAGE_SIZE aligned MR" ? Then patches become >> RDMA/rxe: RDMA/rxe: don't allow registering !PAGE_SIZE aligned MR >> RDMA/rxe: set RXE_PAGE_SIZE_CAP to starting from PAGE_SIZE >> > > What do you take rdma maillist for? Your bugzilla, jira? or your dev > program launch? Or your play ground? May i know which bug you are concerning. Actually i always cannot get your point. > >> Or just keep we have done in the V1 >> >> Thanks >> >> >>> >>> Good Luck. >>> >>> Zhu Yanjun >>> >>>> >>>> I get totally confused now. >>>> >>>> >>>> >>>>> Zhu Yanjun >>>>> >>>>>> >>>>>> [1] https://lore.kernel.org/all/CAHj4cs9XRqE25jyVw9rj9YugffLn5+f=1znaBEnu1usLOciD+g@xxxxxxxxxxxxxx/T/ >>>>>> >>>>>> Li Zhijian (6): >>>>>> RDMA/rxe: RDMA/rxe: don't allow registering !PAGE_SIZE mr >>>>>> RDMA/rxe: set RXE_PAGE_SIZE_CAP to PAGE_SIZE >>>>>> RDMA/rxe: remove unused rxe_mr.page_shift >>>>>> RDMA/rxe: Use PAGE_SIZE and PAGE_SHIFT to extract address from >>>>>> page_list >>>>>> RDMA/rxe: cleanup rxe_mr.{page_size,page_shift} >>>>>> RDMA/rxe: Support PAGE_SIZE aligned MR >>>>>> >>>>>> drivers/infiniband/sw/rxe/rxe_mr.c | 80 ++++++++++++++++----------- >>>>>> drivers/infiniband/sw/rxe/rxe_param.h | 2 +- >>>>>> drivers/infiniband/sw/rxe/rxe_verbs.h | 9 --- >>>>>> 3 files changed, 48 insertions(+), 43 deletions(-) >>>>>> >>>
