On Thu, Feb 23, 2017 at 01:49:47PM +0000, Amrani, Ram wrote: > Thanks, Jason, for this work. > > Per this change, I don't see any need in using mmio_wc_start() in > qedr. The required pattern when dealing with WC memory is this: mmio_wc_start(); writel_to_wc(..); mmio_flush_writes(); On x86-64 this translates to sfence mov .. sfence As far as I know, on x86-64 both sfences are required to ensure that all stores are visible to the device in strictly program order. x86-64 does not strongly order accesses to WC relative to other memory types without SFENCE. mmio_wc_start() is defined to be a superset of the functionality of udma_to_device_barrier() - and it is given a unique name to make it very clear where to use it: directly before writing to WC memory. > wmb() is invoked to make sure that the CPU doesn't issue the > doorbell before the data was flushed (that's why I expected its > implementation to be as the kernel's i.e. sfence. See e-mail Exactly. mmio_wc_start() is defined as the barrier to do that. It specifically guarentees that all writes to C and UC memory are ordered before upcoming writes to WC memory. It is SFENCE on x86-64, which as you say, is what you expected in the first place, so this fixes a bug :\.. FWIW the #define mmio_wc_start() mmio_flush_writes() Is just a stand in that happens to get a safe instruction on every arch. If required each of the macros could have a unique implementation if the arch requires it, however in most cases I would expect that udma_to_device_barrier(), mmio_flush_writes() and mmio_wc_start() are the same instruction. The use of different names is a purely descriptive to aid in understanding what the barriers do and when to use them. Does that address your concern? Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
