On Wed, Dec 16, 2020 at 06:56:11PM +0100, Marek Vasut wrote: > On 12/14/20 9:38 PM, Bjorn Helgaas wrote: > > On Tue, Dec 08, 2020 at 07:05:09PM +0100, Marek Vasut wrote: > > > On 12/8/20 5:40 PM, Bjorn Helgaas wrote: > > > > > > Does this problem occur in both these cases? > > > > > > > > 1) When ASPM enters L1, and > > > > > > > > 2) When software writes PCI_PM_CTRL to put the device in D3hot? > > > > > > > > IIUC both cases require the link to go to L1. I guess the same > > > > software workaround applies to both cases? > > > > > > Yes > > > > If ASPM puts the Link in L1 and the device needs to DMA, how does the > > Link get back to L0? > > It cannot, so I would expect the DMA access would fail. I think that means we cannot enable ASPM L1 at all on this device. I don't think devices or drivers are prepared to deal with this sort of DMA failure. At least, if there is a mechanism for dealing with it, I don't know what it is. Preventing use of ASPM L1 probably means some sort of quirk to override whatever the controller advertises in its Link Capabilities register. The software-controlled PCI-PM model (where software writes to the PCI_PM_CTRL register) is different, and it may still be possible to use L1 then. If software puts the device in D1, D2, or D3hot, the device cannot initiate DMA. If it needs to return to D0, it would have to use the PME mechanism, so there is an opportunity for the software workaround. > > Do we use the same data abort hook? If getting > > back to L0 requires help from software, it seems like that would > > invalidate the L1 exit latency advertised by the devices. Wouldn't > > that mean we couldn't safely enable L1 at all unless the endpoint > > could tolerate unlimited exit latency? > > Possibly, there could be limitations to the L1 support in some corner cases. > Does that mean the L1 support should be disabled completely ? The L1 exit latency only applies to the ASPM case. It sounds like we will have to disable L1 for ASPM. But the exit latency doesn't apply to the PCI-PM model where software will explicitly return the device to D0, and the device should not initiate a transaction until it sees the link back in L0. Bjorn
