On Tue, May 17, 2022 at 08:04:58PM +0000, Long Li wrote: > > Subject: Re: [PATCH 05/12] net: mana: Set the DMA device max page size > > > > On Tue, May 17, 2022 at 07:32:51PM +0000, Long Li wrote: > > > > Subject: Re: [PATCH 05/12] net: mana: Set the DMA device max page > > > > size > > > > > > > > On Tue, May 17, 2022 at 02:04:29AM -0700, longli@xxxxxxxxxxxxxxxxx > > wrote: > > > > > From: Long Li <longli@xxxxxxxxxxxxx> > > > > > > > > > > The system chooses default 64K page size if the device does not > > > > > specify the max page size the device can handle for DMA. This do > > > > > not work well when device is registering large chunk of memory in > > > > > that a large page size is more efficient. > > > > > > > > > > Set it to the maximum hardware supported page size. > > > > > > > > For RDMA devices this should be set to the largest segment size an > > > > ib_sge can take in when posting work. It should not be the page size > > > > of MR. 2M is a weird number for that, are you sure it is right? > > > > > > Yes, this is the maximum page size used in hardware page tables. > > > > As I said, it should be the size of the sge in the WQE, not the "hardware page > > tables" > > This driver uses the following code to figure out the largest page > size for memory registration with hardware: > > page_sz = ib_umem_find_best_pgsz(mr->umem, PAGE_SZ_BM, iova); > > In this function, mr->umem is created with ib_dma_max_seg_size() as > its max segment size when creating its sgtable. > > The purpose of setting DMA page size to 2M is to make sure this > function returns the largest possible MR size that the hardware can > take. Otherwise, this function will return 64k: the default DMA > size. As I've already said, you are supposed to set the value that limits to ib_sge and *NOT* the value that is related to ib_umem_find_best_pgsz. It is usually 2G because the ib_sge's typically work on a 32 bit length. Jason
