On Wed, Aug 25, 2010 at 12:40:00AM -0700, Kanoj Sarcar wrote: ... > > Yes, it will make sure one can mask MSI and not drop the > > MSI signal. > > I don't believe it provides any sort of barrier. In flight > > MSIs > > still need to be dealt with on the host side. > > Agreed about in flight msi's: chipset/platform specific code > needs to synchronize across multiple cpu's if there is a > strict guarantee requirement. > > But is it acceptable to have the device send out an msix > after having responded back to the host's entry mask read? No. I agree that violates the portions of the PCIE spec already quoted in this thread. ... > In general, I agree. I am under the assumption though that > Linux handles misbehaving devices that generate spurious > interrupts by turning them off after a while. MSI is the equivalent to "Edge Triggered". Another interrupt should not be sent until the previous one is ACK'd. IIRC, the "spurious" interrupt code waits until 100K interrupts have been sent (or something in that order of magnitude). > So, I think > each port is taking care of handling such interrupts by > potentially maintaining state whether the vector should > trigger or not, which is checked before invoking driver isr. Ok - the generic interrupt code is providing that. That is exactly what I meant needs to be handled. I was thinking the context is some random driver is trying to mask a device's MSI for some reason. > > > > > I suspect any code that mucks with interrupt state would > > also depend > > on the interrupt not triggering at that moment. > > > > At least from the part that I understand, I agree to the above. > When x86 receives an msix, it masks the entry and reads the > mask back on the interrupt cpu. The interrupt is not > triggering at this point. Which begs the question: why readback > at this point? I suspect to limit the window a "spurious" interrupt would get sent by the device. If we are protecting from that case anyway, I think you are probably right that it's ok to drop the flush. > The part that I don't understand is how intercpu synchronization > is achieved in irq rebalancing case, device deinit case, etc. It's been 5+ years since I've look at the IRQ "rebalancing" (interrupt migration) sequence. I don't recall enough to be useful. Device devinit should generate NO interrupts - that's a bug in the device driver if the device is generating interrupts before the IRQ or MSI is initialized and can handle them. Device driver module unload is the other time I've seen ugly race conditions (between interrupts and DMA unmapping mostly). > Does Linux currently take a strict approach to barriers in > these cases, or is it a loose approach where a laggard instance > of the interrupt can still creep to an unintended cpu? In the > loose approach case, the readback does not make much sense to > me. I don't know offhand. In general, I believe stricter approaches help keep the systems more robust in the face of other issues. Race conditions around interrupt handling are pretty hard to debug. Removing this MMIO read needs more careful analysis than I'm willing to put into it. Both to justify the effort (some measurable performance benefit) and "prove" it will work under all conditions. hth, grant -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
