On Tue, Oct 22, 2024 at 11:24:31AM +0100, Catalin Marinas wrote: > > > We should not allow smaller than cache line alignment on architectures > > > that are not cache coherent indeed. > > Even on architectures that are not fully coherent, the coherency is a > property of the device. You may need to somehow pass this information in > struct queue_limits if you want it to be optimal. Well, devices set the queue limits. So this would be a fix in the drivers that set the queue limits. SCSI already does this in the midlayer code, so the main places to fix are nvme und ublk. I cant take care of nvme by copying the scsi pattern. > That said, the DMA debug code also uses the static L1_CACHE_SHIFT and it > will trigger the warning anyway. Some discussion around the DMA API > debug came up during the small ARCH_KMALLOC_MINALIGN changes (don't > remember it was in private with Robin or on the list). Now kmalloc() can > return a small buffer (less than a cache line) that won't be bounced if > the device is coherent (see dma_kmalloc_safe()) but the DMA API debug > code only checks for direction == DMA_TO_DEVICE, not > dev_is_dma_coherent(). For arm64 I did not want to disable small > ARCH_KMALLOC_MINALIGN if CONFIG_DMA_API_DEBUG is enabled as this would > skew the testing by forcing all allocations to be ARCH_DMA_MINALIGN > aligned. > > Maybe I'm missing something in those checks but I'm surprised that the > DMA API debug code doesn't complain about small kmalloc() buffers on x86 > (which never had any bouncing for this specific case since it's fully > coherent). I suspect people just don't enable DMA debugging on x86 for > such devices (typically USB drivers have this issue). I don't think there's too many of these indeed. > So maybe the DMA API debug should have two modes: a generic one that > catches alignments irrespective of the coherency of the device and > another that's specific to the device/architecture coherency properties. > The former, if enabled, should also force a higher minimum kmalloc() > alignment and a dma_get_cache_alignment() > 1. Sounds reasonable.
