On 4.06.2024 4:40 PM, Will Deacon wrote: > On Thu, May 16, 2024 at 01:55:26PM -0500, Andrew Halaney wrote: >> On Thu, May 16, 2024 at 08:20:05PM GMT, Akhil P Oommen wrote: >>> On Thu, May 16, 2024 at 08:15:34AM -0500, Andrew Halaney wrote: >>>> If I understand correctly, you don't need any memory barrier. >>>> writel()/readl()'s are ordered to the same endpoint. That goes for all >>>> the reordering/barrier comments mentioned below too. >>>> >>>> device-io.rst: >>>> >>>> The read and write functions are defined to be ordered. That is the >>>> compiler is not permitted to reorder the I/O sequence. When the ordering >>>> can be compiler optimised, you can use __readb() and friends to >>>> indicate the relaxed ordering. Use this with care. >>>> >>>> memory-barriers.txt: >>>> >>>> (*) readX(), writeX(): >>>> >>>> The readX() and writeX() MMIO accessors take a pointer to the >>>> peripheral being accessed as an __iomem * parameter. For pointers >>>> mapped with the default I/O attributes (e.g. those returned by >>>> ioremap()), the ordering guarantees are as follows: >>>> >>>> 1. All readX() and writeX() accesses to the same peripheral are ordered >>>> with respect to each other. This ensures that MMIO register accesses >>>> by the same CPU thread to a particular device will arrive in program >>>> order. >>>> >>> >>> In arm64, a writel followed by readl translates to roughly the following >>> sequence: dmb_wmb(), __raw_writel(), __raw_readl(), dmb_rmb(). I am not >>> sure what is stopping compiler from reordering __raw_writel() and __raw_readl() >>> above? I am assuming iomem cookie is ignored during compilation. >> >> It seems to me that is due to some usage of volatile there in >> __raw_writel() etc, but to be honest after reading about volatile and >> some threads from gcc mailing lists, I don't have a confident answer :) >> >>> >>> Added Will to this thread if he can throw some light on this. >> >> Hopefully Will can school us. > > The ordering in this case is ensured by the memory attributes used for > ioremap(). When an MMIO region is mapped using Device-nGnRE attributes > (as it the case for ioremap()), the "nR" part means "no reordering", so > readX() and writeX() to that region are ordered wrt each other. > > Note that guarantee _doesn't_ apply to other flavours of ioremap(), so > e.g. ioremap_wc() won't give you the ordering. > > Hope that helps, Just to make sure I'm following, would mapping things as nGnRnE effectively get rid of write buffering, perhaps being a way of debugging whether that in particular is causing issues (at the cost of speed)? Konrad
