From: Robin Murphy <robin.murphy@xxxxxxx> Date: Mon, 29 Jan 2024 14:29:43 +0000 > On 2024-01-29 2:07 pm, Alexander Lobakin wrote: >> From: Alexander Lobakin <aleksander.lobakin@xxxxxxxxx> >> Date: Fri, 26 Jan 2024 17:45:11 +0100 >> >>> From: Robin Murphy <robin.murphy@xxxxxxx> >>> Date: Fri, 26 Jan 2024 15:48:54 +0000 >>> >>>> On 26/01/2024 1:54 pm, Alexander Lobakin wrote: >>>>> From: Eric Dumazet <edumazet@xxxxxxxxxx> >>>>> >>>>> Quite often, NIC devices do not need dma_sync operations on x86_64 >>>>> at least. >>>>> Indeed, when dev_is_dma_coherent(dev) is true and >>>>> dev_use_swiotlb(dev) is false, iommu_dma_sync_single_for_cpu() >>>>> and friends do nothing. >>>>> >>>>> However, indirectly calling them when CONFIG_RETPOLINE=y consumes >>>>> about >>>>> 10% of cycles on a cpu receiving packets from softirq at ~100Gbit >>>>> rate. >>>>> Even if/when CONFIG_RETPOLINE is not set, there is a cost of about 3%. >>>>> >>>>> Add dev->skip_dma_sync boolean which is set during the device >>>>> initialization depending on the setup: dev_is_dma_coherent() for >>>>> direct >>>>> DMA, !(sync_single_for_device || sync_single_for_cpu) or positive >>>>> result >>>>> from the new callback, dma_map_ops::can_skip_sync for non-NULL DMA >>>>> ops. >>>>> Then later, if/when swiotlb is used for the first time, the flag >>>>> is turned off, from swiotlb_tbl_map_single(). >>>> >>>> I think you could probably just promote the dma_uses_io_tlb flag from >>>> SWIOTLB_DYNAMIC to a general SWIOTLB thing to serve this purpose now. >>> >>> Nice catch! >> >> BTW, this implies such hotpath check: >> >> if (dev->dma_skip_sync && !READ_ONCE(dev->dma_uses_io_tlb)) >> // ... >> >> This seems less effective than just resetting dma_skip_sync on first >> allocation. > > Well, my point is not to have a dma_skip_sync at all; I'm suggesting the > check would be: > > if (dev_is_dma_coherent(dev) && dev_uses_io_tlb(dev)) > ... Aaah, okay, but what about dma_map_ops? It would then become: if ((!dev->dma_ops || (!dev->dma_ops->sync_single_for_device && !dev->dma_ops->sync_single_for_cpu)) || (dev->dma_ops->flags & DMA_F_SKIP_SYNC)) && dev_is_dma_coherent(dev) && !dev_uses_io_tlb(dev)) dma_sync_ ... Quite a lot and everything except dev_uses_io_tlb() is known at device probing time, that's why I decided to cache the result into a new flag... > > where on the platform which cares about this most, that first condition > is a compile-time constant (and as implied, the second would want to be > similarly wrapped for !SWIOTLB configs). > > Thanks, > Robin. Thanks, Olek
