>Subject: Re: [PATCH v2 rdma-next 1/5] RDMA/umem: Add API to find best driver >supported page size in an MR > >On Tue, Apr 30, 2019 at 01:36:14PM +0000, Saleem, Shiraz wrote: > >> >If we make a MR with VA 0x6400FFF and length 2M-4095 I expect the 2M >> >page size with the NIC. >> > >> >We will have dma_addr_start = 0x2600000 and sg_dma_len = 0x200000 as >> >the SGL is always rounded to pages >> >> why isn't it the sg_dma_len 2M-4095? Is it because we compute npages >> in ib_umem_get() to build the SGL? Could using (addr & PAGE_MASK) and >> then adding dma_len help take care of this? > >We always round to page sizes when building the SGL, so the start is rounded >down and the end remains the same. > >> >I have a feeling the uvirt_addr should be computed more like >> > >> > if (flags & IB_UMEM_VA_BASED_OFFSET) >> > mask |= uvirt_addr ^ umem->addr; >> >> I am not following. >> >> For a case like below where uvirt_addr = umem_addr, mask = 0 and we >> run rdma_find_pgbit on it we ll pick 4K instead of 2M which is wrong. > >> uvirt_addr [0x7f2b98200000] >> best_pgsz [0x201000] >> umem->address [0x7f2b98200000] >> dma_addr_start [0x130e200000] >> dma_addr_end [0x130e400000] >> sg_dma_len [2097152] > >?? > >0x7f2b98200000 ^ 0x7f2b98200000 = 0 > >So mask isn't altered by the requested VA and you get 2M pages. I am still missing the point. mask was initialized to 0 and if we just do "mask |= uvirt_addr ^ umem->addr" for the first SGE, then you ll always get 0 for mask (and one page size) when uvirt_addr = umem->addr irrespective of their values. Maybe 'mask' is a wrong variable name to use but the intention of the algo is to OR together the dma_addr_start for every sg but the first, and the dma_addr_end of every sg but the last. mask is initialized to 0 and tracks the computed value after running through all SG's. Basically mask will track the lowest set bit from the entire set and running rdma_find_pg_bit gets the best page size to use.
