On Fri, Mar 12, 2021 at 8:25 AM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
>
> On Thu, Mar 11, 2021 at 06:41:43PM +0800, Zhu Yanjun wrote:
> > On Mon, Mar 8, 2021 at 8:16 PM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
> > >
> > > On Mon, Mar 08, 2021 at 06:13:52PM +0800, Zhu Yanjun wrote:
> > >
> > > > And I delved into the source code of __sg_alloc_table_from_pages. I
> > > > found that this function is related with ib_dma_max_seg_size. So
> > > > when ib_dma_max_seg_size is set to UINT_MAX, the sg dma address is
> > > > 4K (one page). When ib_dma_max_seg_size is set to SZ_2M, the sg dma
> > > > address is 2M now.
> > >
> > > That seems like a bug, you should fix it
> >
> > Hi, Jason && Leon
> >
> > I compared the function __sg_alloc_table_from_pages with ib_umem_add_sg_table.
> > In __sg_alloc_table_from_pages:
> >
> > "
> > 449 if (prv) {
> > 450 unsigned long paddr = (page_to_pfn(sg_page(prv))
> > * PAGE_SIZE +
> > 451 prv->offset + prv->length) /
> > 452 PAGE_SIZE;
> > 453
> > 454 if (WARN_ON(offset))
> > 455 return ERR_PTR(-EINVAL);
> > 456
> > 457 /* Merge contiguous pages into the last SG */
> > 458 prv_len = prv->length;
> > 459 while (n_pages && page_to_pfn(pages[0]) == paddr) {
> > 460 if (prv->length + PAGE_SIZE > max_segment)
> > 461 break;
> > 462 prv->length += PAGE_SIZE;
> > 463 paddr++;
> > 464 pages++;
> > 465 n_pages--;
> > 466 }
> > 467 if (!n_pages)
> > 468 goto out;
> > 469 }
> >
> > "
> > if prv->length + PAGE_SIZE > max_segment, then set another sg.
> > In the commit "RDMA/umem: Move to allocate SG table from pages",
> > max_segment is dma_get_max_seg_size.
> > Normally it is UINT_MAX. So in my host, prv->length + PAGE_SIZE is
> > usually less than max_segment
> > since length is unsigned int.
>
> I don't understand what you are trying to say
>
> 460 if (prv->length + PAGE_SIZE > max_segment)
>
> max_segment should be a very big number and "prv->length + PAGE_SIZE" should
> always be < max_segment so it should always be increasing the size of
> prv->length and 'rpv' here is the sgl.
Sure.
When max_segment is UINT_MAX, prv->length is UINT_MAX - PAGE_SIZE.
It is big. That is, segment size is UINT_MAX - PAGE_SIZE.
But from the function ib_umem_add_sg_table, the prv->length is SZ_2M.
That is, segment size if SZ_2M.
It is the difference between the 2 functions.
Zhu Yanjun
>
> The other loops are the same.
>
> Jason
