> -----Original Message-----
> From: Lucas Stach <l.stach@xxxxxxxxxxxxxx>
> Sent: 2022年6月9日 15:55
> To: Hongxing Zhu <hongxing.zhu@xxxxxxx>; bhelgaas@xxxxxxxxxx;
> robh+dt@xxxxxxxxxx; broonie@xxxxxxxxxx; lorenzo.pieralisi@xxxxxxx;
> jingoohan1@xxxxxxxxx; festevam@xxxxxxxxx;
> francesco.dolcini@xxxxxxxxxxx
> Cc: linux-pci@xxxxxxxxxxxxxxx; linux-arm-kernel@xxxxxxxxxxxxxxxxxxx;
> linux-kernel@xxxxxxxxxxxxxxx; kernel@xxxxxxxxxxxxxx; dl-linux-imx
> <linux-imx@xxxxxxx>
> Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is
> down
>
> Am Donnerstag, dem 09.06.2022 um 06:17 +0000 schrieb Hongxing Zhu:
> > > -----Original Message-----
> > > From: Lucas Stach <l.stach@xxxxxxxxxxxxxx>
> > > Sent: 2022年6月8日 15:35
> > > To: Hongxing Zhu <hongxing.zhu@xxxxxxx>; bhelgaas@xxxxxxxxxx;
> > > robh+dt@xxxxxxxxxx; broonie@xxxxxxxxxx; lorenzo.pieralisi@xxxxxxx;
> > > jingoohan1@xxxxxxxxx; festevam@xxxxxxxxx;
> > > francesco.dolcini@xxxxxxxxxxx
> > > Cc: linux-pci@xxxxxxxxxxxxxxx;
> > > linux-arm-kernel@xxxxxxxxxxxxxxxxxxx;
> > > linux-kernel@xxxxxxxxxxxxxxx; kernel@xxxxxxxxxxxxxx; dl-linux-imx
> > > <linux-imx@xxxxxxx>
> > > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators
> > > after link is down
> > >
> > > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu:
> > > > Since i.MX PCIe doesn't support hot-plug, reduce power consumption
> > > > as much as possible by disabling clocks and regulators and
> > > > returning error when the link is down.
> > > >
> > > > Signed-off-by: Richard Zhu <hongxing.zhu@xxxxxxx>
> > > > ---
> > > > drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++----
> > > > 1 file changed, 15 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c
> > > > b/drivers/pci/controller/dwc/pci-imx6.c
> > > > index 3ce3993d5797..d122c12193a6 100644
> > > > --- a/drivers/pci/controller/dwc/pci-imx6.c
> > > > +++ b/drivers/pci/controller/dwc/pci-imx6.c
> > > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie
> > > > *pci)
> > > > /* Start LTSSM. */
> > > > imx6_pcie_ltssm_enable(dev);
> > > >
> > > > - dw_pcie_wait_for_link(pci);
> > > > + ret = dw_pcie_wait_for_link(pci);
> > > > + if (ret)
> > > > + goto err_out;
> > >
> > > This adds back error handling that has been intentionally removed in
> > > f81f095e8771 ("PCI: imx6: Allow to probe when
> > > dw_pcie_wait_for_link() fails").
> > > While I agree that disabling the clocks and regulators is the right
> > > thing to do when we don't manage to get a link, we should still
> > > allow the driver to probe, so please add a "ret = 0" to this newly
> > > added non-fatal error paths.
> > >
> > Thanks for your review comments.
> > There would be a long latency if the link is down and probe is
> > finished
> > successfully.
> > Since the dw_pcie_wait_for_link() would be invoked twice in every
> > driver probe
> > and resume operation later. Each dw_pcie_wait_for_link() would
> > consume about
> > 90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency
> > would
> > bring bad user experience.
> >
> > Here are the logs when probe is allowed when PCIe link is down:
> > [ 55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling
> >
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> > 8 @ 1835, parent: bus@5f000000
> > ...
> > [ 56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came
> > up
> > [ 57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came
> > up
> > ...
> > [ 57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM:
> >
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> > 8 returned 0 after 2136334 usecs
> >
> > [ 57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling
> >
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> > 8 @ 1835, parent: bus@5f000000
> > ...
> > [ 58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came
> > up
> > [ 59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came
> > up
> > ...
> > [ 59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM:
> >
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> > 8 returned 0 after 2135949 usecs
> >
> > So, I'm prefer that it's better to let the probe failed when link is
> > down.
> > How do you think about that?
>
> It different from the behavior of other platforms, that still show the root bridge
> in lspci, even if the link is down. I've seen people confused by this behavior.
> Come to think about it: does lspci work when all the clocks are disabled?
>
> The latency in the probe path is not that relevant, as it is done asynchronous,
> so it doesn't stall the boot process. You have a point on the suspend/resume
> path however. Maybe we need to remember the link state in suspend to know
> if resume should even do anything useful?
>
This sounds good. Thus, one flag is required to specify the link is up or not
before suspend. If the link is down, the imx6_pcie_start_link() shouldn't
invoked at all when resume back.
Then we can avoid the long latency and keep probe finished when link is down.
In this scenario, we can let probe successfully, although the link is down.
And the last patch can be dropped, since the clocks/regulators are always on
in this case.
Thanks for your suggestion and inspiration.
Best Regards
Richard Zhu
> Regards,
> Lucas
>
> >
> > > >
> > > > if (pci->link_gen == 2) {
> > > > /* Allow Gen2 mode after the link is up. */ @@ -
> > > > 876,12 +878,14
> > > @@
> > > > static int imx6_pcie_start_link(struct dw_pcie *pci)
> > > > ret =
> > > > imx6_pcie_wait_for_speed_change(imx6_pcie);
> > > > if (ret) {
> > > > dev_err(dev, "Failed to bring link up!\n");
> > > > - goto err_reset_phy;
> > > > + goto err_out;
> > > > }
> > > > }
> > > >
> > > > /* Make sure link training is finished as well!
> > > > */
> > > > - dw_pcie_wait_for_link(pci);
> > > > + ret = dw_pcie_wait_for_link(pci);
> > > > + if (ret)
> > > > + goto err_out;
> > > > } else {
> > > > dev_info(dev, "Link: Gen2 disabled\n");
> > > > }
> > > > @@ -890,11 +894,18 @@ static int imx6_pcie_start_link(struct
> > > > dw_pcie
> > > *pci)
> > > > dev_info(dev, "Link up, Gen%i\n", tmp & PCI_EXP_LNKSTA_CLS);
> > > > return 0;
> > > >
> > > > -err_reset_phy:
> > > > +err_out:
> > > > dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n",
> > > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0),
> > > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1));
> > > > imx6_pcie_reset_phy(imx6_pcie);
> > > > + imx6_pcie_clk_disable(imx6_pcie);
> > > > + if (imx6_pcie->phy != NULL) {
> > >
> > > Please use the more common if (imx6_pcie->phy) here.
> > >
> > Okay. Thanks.
> >
> > Best Regards
> > Richard Zhu
> >
> > > Regards,
> > > Lucas
> > >
> > > > + phy_power_off(imx6_pcie->phy);
> > > > + phy_exit(imx6_pcie->phy);
> > > > + }
> > > > + if (imx6_pcie->vpcie)
> > > > + regulator_disable(imx6_pcie->vpcie);
> > > > return ret;
> > > > }
> > > >
> > >
> >
>
