On Thu, Oct 15, 2020 at 09:31:27PM -0300, Jason Gunthorpe wrote:
> On Thu, Oct 15, 2020 at 03:21:34PM +0300, Maor Gottlieb wrote:
> >
> > On 10/15/2020 2:23 PM, Gal Pressman wrote:
> > > On 15/10/2020 10:44, Maor Gottlieb wrote:
> > > > On 10/15/2020 1:51 AM, Jason Gunthorpe wrote:
> > > > > On Tue, Oct 13, 2020 at 09:33:14AM -0500, Bob Pearson wrote:
> > > > > > Jason,
> > > > > >
> > > > > > Just pulled for-next and now hit the following warning.
> > > > > > Register user space memory is not longer working.
> > > > > > I am trying to debug this but if you have any idea where to look let me know.
> > > > > The offset_in_page is wrong, but it is protecting some other logic..
> > > > >
> > > > > Maor? Leon? Can you sort it out tomorrow?
> > > > Leon and I investigated it. This check existed before my series to protect the
> > > > alloc_table_from_pages logic. It's still relevant.
> > > > This patch that broke it: 54816d3e69d1 ("RDMA: Explicitly pass in the
> > > > dma_device to ib_register_device"), and according to below link it was
> > > > expected. The safest approach is to set the max_segment_size back the 2GB in
> > > > all drivers. What do you think?
> > > >
> > > > https://lore.kernel.org/linux-rdma/20200923072111.GA31828@xxxxxxxxxxxxx/
> > > FWIW, EFA is broken as well (same call trace) so it's not just software drivers.
> >
> > This is true to all drivers that call to ib_umem_get and set UINT_MAX as
> > max_segment_size.
> > Jason, maybe instead of set UINT_MAX as max_segment_size, need to set
> > SCATTERLIST_MAX_SEGMENT which does the required alignment.
>
> SCATTERLIST_MAX_SEGMENT is almost never used, however there are lots
> of places passing UINT_MAX or similar as the max_segsize for DMA.
>
> The only place that does use it looks goofy to me:
>
> dma_set_max_seg_size(dev->dev, min_t(unsigned int, U32_MAX & PAGE_MASK,
> SCATTERLIST_MAX_SEGMENT));
>
> The seg_size should reflect the HW capability, not be mixed in with
> knowledge about SGL internals. If the SGL can't build up to the HW
> limit then it is fine to internally silently reduce it.
>
> So I think we need to fix the scatterlist code, like below, and
> just remove SCATTERLIST_MAX_SEGMENT completely.
>
> It fixes things? Are you OK with this Christoph?
>
> I need to get this fixed for the merge window PR I want to send on
> Friday.
This looks ok. Not critical, but I think we should follow up and kill
off the confusing SCATTERLIST_MAX_SEGMENT as well soon
