The behaviour of mips_dma_unmap_page() & mips_dma_unmap_sg() with
regards to cache maintenance differ for no good reason. Whilst
mips_dma_unmap_page() correctly takes into account whether a CPU may
have speculatively prefetched data into caches by using
cpu_needs_post_dma_flush(), it ignores the direction of the DMA transfer
& thus performs cache maintenance after DMA_TO_DEVICE transfers for no
good reason. Meanwhile mips_dma_unmap_sg() avoids unnecessary cache
maintenance for DMA_TO_DEVICE transfers but performs unnecessary cache
maintenance on CPUs which cannot have speculatively fetched data into
the caches.
Fix this by using the same condition for cache maintenance in both
mips_dma_unmap_page() & mips_dma_unmap_sg(). We perform cache
maintenance when unmapping if and only if both:
- cpu_needs_post_dma_flush() returns true, indicating that the device
performing DMA is not cache-coherent and the CPU may speculatively
prefetch data from memory being DMAed to.
- The direction of the DMA is not DMA_TO_DEVICE, meaning that the
device may have written to memory & we should invalidate our cached
view in order to observe whatever the device wrote.
Signed-off-by: Paul Burton <paul.burton@xxxxxxxxxx>
Cc: Ralf Baechle <ralf@xxxxxxxxxxxxxx>
Cc: linux-mips@xxxxxxxxxxxxxx
---
arch/mips/mm/dma-default.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index 46d5696..53b00a1 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -293,7 +293,7 @@ static inline void __dma_sync(struct page *page,
static void mips_dma_unmap_page(struct device *dev, dma_addr_t dma_addr,
size_t size, enum dma_data_direction direction, unsigned long attrs)
{
- if (cpu_needs_post_dma_flush(dev))
+ if (cpu_needs_post_dma_flush(dev) && direction != DMA_TO_DEVICE)
__dma_sync(dma_addr_to_page(dev, dma_addr),
dma_addr & ~PAGE_MASK, size, direction);
plat_post_dma_flush(dev);
@@ -338,7 +338,7 @@ static void mips_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
struct scatterlist *sg;
for_each_sg(sglist, sg, nhwentries, i) {
- if (!plat_device_is_coherent(dev) &&
+ if (cpu_needs_post_dma_flush(dev) &&
direction != DMA_TO_DEVICE)
__dma_sync(sg_page(sg), sg->offset, sg->length,
direction);
--
2.10.2
