On 5/11/2023 8:14 AM, Ilpo Järvinen wrote:
> Don't assume that only the driver would be accessing LNKCTL/LNKCTL2.
> ASPM policy changes can trigger write to LNKCTL outside of driver's
> control. And in the case of upstream (parent), the driver does not even
> own the device it's changing the registers for.
>
> Use pcie_lnkctl_clear_and_set() and pcie_lnkctl2_clear_and_set() which
> do proper locking to avoid losing concurrent updates to the register
> value.
>
> Suggested-by: Lukas Wunner <lukas@xxxxxxxxx>
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@xxxxxxxxxxxxxxx>
> ---
> drivers/infiniband/hw/hfi1/aspm.c | 16 ++++++----------
> drivers/infiniband/hw/hfi1/pcie.c | 28 ++++++----------------------
> 2 files changed, 12 insertions(+), 32 deletions(-)
>
> diff --git a/drivers/infiniband/hw/hfi1/aspm.c b/drivers/infiniband/hw/hfi1/aspm.c
> index a3c53be4072c..d3f3b7e9b833 100644
> --- a/drivers/infiniband/hw/hfi1/aspm.c
> +++ b/drivers/infiniband/hw/hfi1/aspm.c
> @@ -66,12 +66,10 @@ static void aspm_hw_enable_l1(struct hfi1_devdata *dd)
> return;
>
> /* Enable ASPM L1 first in upstream component and then downstream */
> - pcie_capability_clear_and_set_word(parent, PCI_EXP_LNKCTL,
> - PCI_EXP_LNKCTL_ASPMC,
> - PCI_EXP_LNKCTL_ASPM_L1);
> - pcie_capability_clear_and_set_word(dd->pcidev, PCI_EXP_LNKCTL,
> - PCI_EXP_LNKCTL_ASPMC,
> - PCI_EXP_LNKCTL_ASPM_L1);
> + pcie_lnkctl_clear_and_set(parent, PCI_EXP_LNKCTL_ASPMC,
> + PCI_EXP_LNKCTL_ASPM_L1);
> + pcie_lnkctl_clear_and_set(dd->pcidev, PCI_EXP_LNKCTL_ASPMC,
> + PCI_EXP_LNKCTL_ASPM_L1);
> }
>
> void aspm_hw_disable_l1(struct hfi1_devdata *dd)
> @@ -79,11 +77,9 @@ void aspm_hw_disable_l1(struct hfi1_devdata *dd)
> struct pci_dev *parent = dd->pcidev->bus->self;
>
> /* Disable ASPM L1 first in downstream component and then upstream */
> - pcie_capability_clear_and_set_word(dd->pcidev, PCI_EXP_LNKCTL,
> - PCI_EXP_LNKCTL_ASPMC, 0x0);
> + pcie_lnkctl_clear_and_set(dd->pcidev, PCI_EXP_LNKCTL_ASPMC, 0);
> if (parent)
> - pcie_capability_clear_and_set_word(parent, PCI_EXP_LNKCTL,
> - PCI_EXP_LNKCTL_ASPMC, 0x0);
> + pcie_lnkctl_clear_and_set(parent, PCI_EXP_LNKCTL_ASPMC, 0);
> }
>
> static void aspm_enable(struct hfi1_devdata *dd)
> diff --git a/drivers/infiniband/hw/hfi1/pcie.c b/drivers/infiniband/hw/hfi1/pcie.c
> index 08732e1ac966..fe7324d38d64 100644
> --- a/drivers/infiniband/hw/hfi1/pcie.c
> +++ b/drivers/infiniband/hw/hfi1/pcie.c
> @@ -1212,14 +1212,10 @@ int do_pcie_gen3_transition(struct hfi1_devdata *dd)
> (u32)lnkctl2);
> /* only write to parent if target is not as high as ours */
> if ((lnkctl2 & PCI_EXP_LNKCTL2_TLS) < target_vector) {
> - lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS;
> - lnkctl2 |= target_vector;
> - dd_dev_info(dd, "%s: ..new link control2: 0x%x\n", __func__,
> - (u32)lnkctl2);
> - ret = pcie_capability_write_word(parent,
> - PCI_EXP_LNKCTL2, lnkctl2);
> + pcie_lnkctl2_clear_and_set(parent, PCI_EXP_LNKCTL2_TLS,
> + target_vector);
You are missing an assignment to "ret" above.
-Dean
External recipient
