On Tue, Jan 30, 2024 at 06:00:51PM +0800, Jian-Hong Pan wrote: > The remmapped PCIe port and NVMe have PCI PM L1 substates capability on > ASUS B1400CEAE, but they are disabled originally: > > Capabilities: [900 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+ > PortCommonModeRestoreTime=32us PortTPowerOnTime=10us > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- > T_CommonMode=0us LTR1.2_Threshold=0ns > L1SubCtl2: T_PwrOn=10us > > Power on all of the VMD remapped PCI devices before enable PCI-PM L1 PM > Substates by following "Section 5.5.4 of PCIe Base Spec Revision 5.0 > Version 0.1". Then, PCI PM's L1 substates control are enabled > accordingly. > +static int vmd_power_on_pci_device(struct pci_dev *pdev, void *userdata) > +{ > + pci_set_power_state(pdev, PCI_D0); As I believe Bjorn already hinted at, this should probably be done in vmd_pm_enable_quirk(). Also, you need to use the new pci_set_power_state_locked() helper since these callbacks are called from pci_walk_bus() with the bus semaphore held: https://lore.kernel.org/lkml/20240130100243.11011-1-johan+linaro@xxxxxxxxxx/ > + return 0; > +} > + > /* > * Enable ASPM and LTR settings on devices that aren't configured by BIOS. > */ > @@ -928,6 +934,13 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) > vmd_acpi_begin(); > > pci_scan_child_bus(vmd->bus); > + > + /* > + * Make PCI devices at D0 when enable PCI-PM L1 PM Substates from > + * Section 5.5.4 of PCIe Base Spec Revision 5.0 Version 0.1 > + */ > + pci_walk_bus(vmd->bus, vmd_power_on_pci_device, NULL); > + > vmd_domain_reset(vmd); > > /* When Intel VMD is enabled, the OS does not discover the Root Ports Johan