On Mon, Dec 9, 2024 at 8:11 PM Christoph Hellwig <hch@xxxxxx> wrote:
>
> On Mon, Dec 09, 2024 at 11:15:02AM -0800, Tim Harvey wrote:
> > After a lot of back and forth and investigation this is due to the
> > IMX8M SoC's not having an IOMMU thus swiotlb is being used and ath11k
> > is requesting some buffers that are too large for swiotlb to provide.
> > There is a specific patch which added the HAL_WBM2SW_RELEASE buffers
> > to cacheable memory that could be reverted to fix this but the concern
> > was that it would impact performance moving those buffers to
> > non-cacheable memory (there are three ~1MiB buffers being allocated):
> > commit d0e2523bfa9cb ("ath11k: allocate HAL_WBM2SW_RELEASE ring from
> > cacheable memory").
>
> The combination of "buffers" and "swiotlb" sounds like Robin was right
> below.
>
> > The chain of events as best I can tell are:
> >
> > commit 6452f0a3d565 ("ath11k: allocate dst ring descriptors from
> > cacheable memory")
> > - Nov 12 2021 (made it into Linux 5.17)
> > - changes allocation of reo_dst rings to cacheable memory to allow
> > cached descriptor access to optimize CPU usage
> > - this is flawed because it uses virt_to_phys() to allocate cacheable
> > memory which does not work on systems with an IOMMU enabled or using
> > software IOMMU (swiotlb); this causes a kernel crash on client
> > association
>
> And this is where it started to take a wrong turn, that everyhing
> later basically made worse. If you have long living and potentially
> large DMA allocations, you need to use dma_alloc_* interfaces.
>
> 5.17 already had dma_alloc_pages for quite a while which was and still is
> the proper interface to use. For much older kernel you'd be stuck
> with dma_alloc_noncoherent or dma_alloc_attrs with the right flag,
> but even that would have been much better.
Christoph,
I'm not clear what you are suggesting be done here. Are you suggesting
that ath11k has been using the wrong mechanism by calling
dma_map_single for cached DMA buffers? I'm not all that familiar with
ath11k so I can't tell what buffers are considered long living.
Best Regards,
Tim
