On Fri, 1 Jul 2005, Alan Cox wrote: > > Is that non-reordering specified anywhere for the API or does it just > > happen to be satisfied by most implementations? Ours (for MIPS, that is) > > for example does nothing to ensure that. > > It is defined by the device I/O document as follows: > > > The read and write functions are defined to be ordered. That is > the > compiler is not permitted to reorder the I/O sequence. When the > ordering can be compiler optimised, you can use <function> > __readb</function> and friends to indicate the relaxed ordering. > Use > this with care. Oh, wonderful! -- another set of three functions per each operation, for direct, CPU-endian and memory-endian accesses. Sigh... > Note order - not synchronicity. On that it says But that mentions compiler only, not CPU ordering! I understand the BIU of the issuing CPU and any external hardware is still permitted to merge/reorder these accesses unless separated by wmb()/rmb()/mb() as appropriate. Note that there are MIPS-based systems that e.g. retrieve data pending in the write-back buffer (which is logically external to the CPU; sometimes even physically) for reads, e.g. with: writel(COMMAND, dev->csr); status = readl(dev->csr); you'll likely get COMMAND in status, rather than any actual value of dev->csr and no read cycle ever reaches that device at all! You need an mb() in between so that COMMAND leaves the CPU domain before issuing a read for this code to work as expected. And of course an arbitrary number of read cycles to dev->irq_status placed after readl() above may bypass the write as well. We have that iob() macro/call as well, so that you can push cycles out of the CPU domain immediately as well, which is equivalent to: mb(); make_host_complete_writes(); > While the basic functions are defined to be synchronous with > respect > to each other and ordered with respect to each other the busses > the > devices sit on may themselves have asynchronicity. In particular > many > authors are burned by the fact that PCI bus writes are posted > asynchronously. A driver author must issue a read from the same > device to ensure that writes have occurred in the specific cases > the > author cares. This kind of property cannot be hidden from driver > writers in the API. In some cases, the read used to flush the > device > may be expected to fail (if the card is resetting, for > example). In > that case, the read should be done from config space, which is > guaranteed to soft-fail if the card doesn't respond. True and obvious once cycles actually reach your I/O bus of choice -- rules for that bus apply from then on. > > What if the host I/O bus is not PCI? For this kind of stuff I tend to > > think in the terms of TURBOchannel systems, just to be sure not to get > > influenced by the most common hardware. ;-) > > The bus behaviour is bus defined. Certainly -- does it apply to host buses as well from the Linux point of view? I don't think drivers should be made aware of them -- they should be abstracted by the means of these barriers. > > Again, the I/O bus your host is attached to need not be PCI and you may > > need a bridge specific operation to make your write be completed, possibly > > combined with your quoted sequence (if there is actually PCI somewhere in > > the system; think AlphaServer 8400). > > We don't currently have cross bridge "io_write_and_be_synchronous()" > type functions. So far drivers have always known what to do. Your > example might break that of course. So far I've been able to get away with that iob() function, but if the bus and buffering hierarchy gets even more complicated, there may be more barriers like this needed. Maciej
