On Wed, Oct 06, 2021 at 12:37:02AM +0000, Kelvin.Cao@xxxxxxxxxxxxx wrote: > On Tue, 2021-10-05 at 15:11 -0500, Bjorn Helgaas wrote: > > On Mon, Oct 04, 2021 at 08:51:06PM +0000, Kelvin.Cao@xxxxxxxxxxxxx > > wrote: > > > On Sat, 2021-10-02 at 10:11 -0500, Bjorn Helgaas wrote: > > > > I *thought* the problem was that the PCIe Memory Read failed > > > > and the Root Complex fabricated ~0 data to complete the CPU > > > > read. But now I'm not sure, because it sounds like it might > > > > be that the PCIe transaction succeeds, but it reads data that > > > > hasn't been updated by the firmware, i.e., it reads 'in > > > > progress' because firmware hasn't updated it to 'done'. > > > > > > The original message was sort of misleading. After a firmware > > > reset, CPU getting ~0 for the PCIe Memory Read doesn't explain > > > the hang. In a MRPC execution (DMA MRPC mode), the MRPC status > > > which is located in the host memory, gets initialized by the CPU > > > and updated/finalized by the firmware. In the situation of a > > > firmware reset, any MRPC initiated afterwards will not get the > > > status updated by the firmware per the reason you pointed out > > > above (or similar, to my understanding, firmware can no longer > > > DMA data to host memory in such cases), therefore the MRPC > > > execution will never end. > > > > I'm glad this makes sense to you, because it still doesn't to me. > > > > check_access() does an MMIO read to something in BAR0. If that > > read returns ~0, it means either the PCIe Memory Read was > > successful and the Switchtec device supplied ~0 data (maybe > > because firmware has not initialized that part of the BAR) or the > > PCIe Memory Read failed and the root complex fabricated the ~0 > > data. > > > > I'd like to know which one is happening so we can clarify the > > commit log text about "MRPC command executions hang indefinitely" > > and "host wil fail all GAS reads." It's not clear whether these > > are PCIe protocol issues or driver/firmware interaction issues. > > I think it's the latter case, the ~0 data was fabricated by the root > complex, as the MMIO read in check_access() always returns ~0 until > a reboot or a rescan happens. If the root complex fabricates ~0, that means a PCIe transaction failed, i.e., the device didn't respond. Rescan only does config reads and writes. Why should that cause the PCIe transactions to magically start working?
