On Wed, Jan 10, 2024 at 07:24:31AM -0800, David E. Box wrote:
> On Thu, 2023-12-28 at 18:30 -0600, Bjorn Helgaas wrote:
> > On Thu, Dec 28, 2023 at 04:31:12PM -0600, Bjorn Helgaas wrote:
> > > On Wed, Dec 20, 2023 at 05:12:50PM -0800, David E. Box wrote:
> > > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > > index 55bc3576a985..3c4b2647b4ca 100644
> > > > --- a/drivers/pci/pci.c
> > > > +++ b/drivers/pci/pci.c
> >
> > > > @@ -1579,7 +1579,7 @@ static void pci_restore_pcie_state(struct pci_dev
> > > > *dev)
> > > > {
> > > > ...
> >
> > > > + So we restore here only the
> > > > + * LNKCTL register with the ASPM control field clear. ASPM will
> > > > + * be restored in pci_restore_aspm_state().
> > > > + */
> > > > + val = cap[i++] & ~PCI_EXP_LNKCTL_ASPMC;
> > > > + pcie_capability_write_word(dev, PCI_EXP_LNKCTL, val);
> > >
> > > When CONFIG_PCIEASPM is not set, we will clear ASPMC here and never
> > > restore it. I don't know if this ever happens. Do we need to worry
> > > about this? Might firmware restore ASPMC itself before we get here?
> > > What do we want to happen in this case?
>
> I just checked this. L1 does get disabled which we don't want. We
> need to save and restore the BIOS ASPM configuration even when
> CONFIG_PCIEASPM is not set.
There's some other ASPM stuff that we want even when CONFIG_PCIEASPM
is not set. I think some of that code is currently in probe.c and
pci.c.
I can't find it right now, but we had some discussion about moving
that code into aspm.c, compiling aspm.c unconditionally, and adding
CONFIG_PCIEASPM ifdefs inside it for these cases. Maybe this is the
time do to that? If so, probably a preliminary patch or two to do
the code movement without any functional changes, followed by the
actual fixes.
> > > Since ASPM is intertwined with the PCIe Capability, can we call
> > > pci_restore_aspm_state() from here instead of from
> > > pci_restore_state()?
> > >
> > > Calling it here would make it easier to see the required ordering
> > > (LNKCTL with ASPMC cleared, restore ASPM L1SS, restore ASPMC) and
> > > it would be obvious that none of the other stuff in
> > > pci_restore_state() is relevant (PASID, PRI, ATS, VC, etc).
> > >
> > > If that could be done, I think it would make sense to do the same with
> > > pci_save_aspm_state() even though it's a little more independent.
>
> Makes sense
>
> > The lspci output in Michael's report at
> > https://lore.kernel.org/r/76c61361-b8b4-435f-a9f1-32b716763d62@xxxxxxxxxxx
> > reminded me that LTR is important for L1.2, and we currently have
> > this:
> >
> > pci_restore_state
> > pci_restore_ltr_state
> > pci_restore_pcie_state
> >
> > I wonder if pci_restore_ltr_state() should be called from
> > pci_restore_pcie_state() as well? It's intimately connected to ASPM,
> > and that connection isn't very clear in the current code.
>
> Make sense too since LTR is a required capability for L1.2. I'll
> send updated patches after the merge window.
Sounds good, will look for them :)
Bjorn
