On Tue, 2010-04-06 at 15:20 +0100, Nick Piggin wrote: > On Tue, Mar 23, 2010 at 10:24:07AM +0000, Catalin Marinas wrote: > > On Mon, 2010-03-22 at 12:02 +0000, Nick Piggin wrote: > > > So IMO, we need to take all these out of lock primitives and just > > > increase awareness of it. Get rid of mmiowb. wmb() should be enough > > > to keep mmio stores inside the store to drop any lock (by definition). > > > > I think we have different scenarios for wmb and mmiowb (my > > understanding). One is when the driver writes to a coherent DMA buffer > > (usually uncached) and it than need to drain the write buffer before > > informing the device to start the transfer. That's where wmb() would be > > used (with normal uncached memory). > > > > The mmiowb() may need to go beyond the CPU write-buffer level into the > > PCI bus etc. but only for relative ordering of the I/O accesses. The > > memory-barriers.txt suggests that mmiowb(). My understanding is that > > mmiowb() drains any mmio buffers while wmb() drains normal memory > > buffers. > > No barriers are defined to drain anything, only order. wmb() is defined > to order all memory stores, so all previous stores cached and IO are > seen before all subsequent stores. And considering that we are talking > about IO, "seen" obviously means seen by the device as well as other > CPUs. Indeed, the barriers aren't defined to drain anything, though they may do it on specific implementations (or when "seen" actually requires draining). The Documentation/DMA-API.txt file mentions that the CPU write buffer may need to be flushed after writing coherent memory but the kernel doesn't define any primitive for doing this. Hence my assumption that this is the job of wmb(). > What is needed is to make the default accessors strongly ordered and > so driver writers can be really dumb about it, and IO / spinlock etc > synchronization "just works". On ARM, the I/O accessors are ordered with respect to device memory accesses but not with respect to normal non-cacheable memory (dma_alloc_coherent). If we want to make the writel etc. accessors ordered with respect to the normal non-cacheable memory, that would be really expensive on several ARM platforms. Apart from the CPU barrier (a full one - DSB - to drain the write buffer), some platforms require draining the write buffer of the L2 cache as well (by writing to other registers to the L2 cache controller). So I'm more in favour of having stronger semantics for wmb() and leaving the I/O accessors semantics to only ensure device memory ordering. -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-arch" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
