[+to Sathy, David in case you want to update your Reviewed-by]
On Thu, Feb 22, 2024 at 11:14:36PM +0530, Vidya Sagar wrote:
> Many PCIe device drivers save the configuration state of their respective
> devices during probe and restore the same when their 'slot_reset' hook
> is called through PCIe Error Recovery Handler.
>
> If the system has a change in ASPM policy after the driver's probe is
> called and before error event occurred, 'slot_reset' hook restores the
> PCIe configuration state to what it was at the time of probe but not to
> what it was just before the occurrence of the error event.
> This effectively leads to a mismatch in the ASPM configuration between
> the device and its upstream parent device.
>
> Update the saved configuration state of the device with the latest info
> whenever there is a change w.r.t ASPM policy.
>
> Signed-off-by: Vidya Sagar <vidyas@xxxxxxxxxx>
> -void pci_save_aspm_state(struct pci_dev *pdev);
> +void pci_save_aspm_l1ss_state(struct pci_dev *pdev);
I rebased this again on top of my pci/aspm updates to remove the need
for the rename above.
> +static void pci_save_aspm_state(struct pci_dev *dev)
> +{
> + struct pci_cap_saved_state *save_state;
> + u16 *cap;
> +
> + if (!pci_is_pcie(dev))
> + return;
> +
> + save_state = pci_find_saved_cap(dev, PCI_CAP_ID_EXP);
> + if (!save_state)
> + return;
> +
> + cap = (u16 *)&save_state->cap.data[0];
> + pcie_capability_read_word(dev, PCI_EXP_LNKCTL, &cap[1]);
And I changed this part so it only updates the PCI_EXP_LNKCTL_ASPMC
bits, not the entire LNKCTL.
Updating the entire saved register probably wouldn't *break* anything,
but it could randomly hide other LNKCTL changes depending on whether
or not ASPM configuration was changed in the interim. For example:
- driver .probe() saves LNKCTL
- LNKCTL changes some non-ASPMC thing via setpci or other mechanism
- save_state updated via pcie_config_aspm_link()
A restore in .slot_reset() would restore different LNKCTL values for
the non-ASPMC change depending on whether pcie_config_aspm_link() was
used.
I applied it on pci/aspm for v6.9. Please take a look and make sure
it still does what you need:
https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/commit/?h=aspm&id=a6315434436d587f70e489e6365c5b7e20176a71
Sathy and David, I didn't add your Reviewed-by because I didn't want
to presume that you were OK with my changes. But I'd be more than
happy to add them if you take a look.
Bjorn
> +}
> +
> void pci_aspm_get_l1ss(struct pci_dev *pdev)
> {
> /* Read L1 PM substate capabilities */
> pdev->l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS);
> }
>
> -void pci_save_aspm_state(struct pci_dev *pdev)
> +void pci_save_aspm_l1ss_state(struct pci_dev *pdev)
> {
> struct pci_cap_saved_state *save_state;
> u16 l1ss = pdev->l1ss;
> @@ -309,10 +325,12 @@ static void pcie_set_clkpm_nocheck(struct pcie_link_state *link, int enable)
> struct pci_bus *linkbus = link->pdev->subordinate;
> u32 val = enable ? PCI_EXP_LNKCTL_CLKREQ_EN : 0;
>
> - list_for_each_entry(child, &linkbus->devices, bus_list)
> + list_for_each_entry(child, &linkbus->devices, bus_list) {
> pcie_capability_clear_and_set_word(child, PCI_EXP_LNKCTL,
> PCI_EXP_LNKCTL_CLKREQ_EN,
> val);
> + pci_save_aspm_state(child);
> + }
> link->clkpm_enabled = !!enable;
> }
>
> @@ -931,6 +949,12 @@ static void pcie_config_aspm_link(struct pcie_link_state *link, u32 state)
> pcie_config_aspm_dev(parent, upstream);
>
> link->aspm_enabled = state;
> +
> + /* Update latest ASPM configuration in saved context */
> + pci_save_aspm_state(link->downstream);
> + pci_save_aspm_l1ss_state(link->downstream);
> + pci_save_aspm_state(parent);
> + pci_save_aspm_l1ss_state(parent);
> }
>
> static void pcie_config_aspm_path(struct pcie_link_state *link)
> --
> 2.25.1
>
