On Tue, Dec 04, 2018 at 02:20:48PM +0300, Mika Westerberg wrote:
> Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is
> connected to an Alpine Ridge Thunderbolt controller. This port has slot
> implemented bit set in the config space but other than that it is not
> hotplug capable in the sense we are expecting in Linux (it has
> dev->is_hotplug_bridge set to 0):
>
> 00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5
> Bus: primary=00, secondary=05, subordinate=46, sec-latency=0
> Memory behind bridge: 78000000-8fffffff [size=384M]
> Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M]
> ...
> Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
> ...
> SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
> Slot #8, PowerLimit 25.000W; Interlock- NoCompl+
> SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
> Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
> SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
> Changed: MRL- PresDet+ LinkState+
>
> This system is using ACPI based hotplug to notify the OS that it needs
> to rescan the PCI bus (ACPI hotplug).
>
> If there is nothing connected in any of the Thunderbolt ports the root
> port will not have any runtime PM active children and is thus
> automatically runtime suspended pretty soon after boot by PCI PM core.
> Now, when a device is connected the BIOS SMI handler responsible for
> enumerating newly added devices is not able to find anything because the
> port is in D3.
Ugh. I don't see how this is a maintainable solution. Are we going
to have to just update this blacklist empirically as we get reports of
systems that are "broken"?
I say "broken" because I don't think we can point to anything here
that doesn't conform to the specs, so maybe we tripped over something
that *should* be covered in the spec, or maybe we're just not
interpreting something correctly.
For example, it looks like PCI_EXP_FLAGS_SLOT is set, but Linux
basically ignores it. Maybe if PCI_EXP_FLAGS_SLOT is set but we
aren't using pciehp, we should assume any hotplug would be handled via
acpiphp? And in that case, we should avoid doing anything that would
prevent platform firmware from enumerating things below the bridge?
> Prevent this from happening by blacklisting PCI power management of this
> particular Gigabyte system.
>
> Reported-by: Kedar A Dongre <kedar.a.dongre@xxxxxxxxx>
Is there a bugzilla or any other URL we could include here to help with
future changes in this area?
> Signed-off-by: Mika Westerberg <mika.westerberg@xxxxxxxxxxxxxxx>
> ---
> I checked booting Windows on the same system and it does not put any of the
> PCIe root ports to low power states so there is no issue in Windows. I'm
> also quite certain Windows does not have similar blacklist.
>
> I wonder if our pci_bridge_d3_possible() heuristics would need to be
> refined somehow? At least if this blacklist starts growing.
>
> drivers/pci/pci.c | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index c9d8e3c837de..1c6e47522c84 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -2501,6 +2501,23 @@ void pci_config_pm_runtime_put(struct pci_dev *pdev)
> pm_runtime_put_sync(parent);
> }
>
> +static const struct dmi_system_id bridge_d3_blacklist[] = {
> + {
> + /*
> + * Gigabyte X299 root port is not marked as hotplug
> + * capable which allows Linux to power manage it.
> + * However, this confuses the BIOS SMI handler so don't
> + * power manage root ports on that system.
> + */
> + .ident = "X299 DESIGNARE EX-CF",
> + .matches = {
> + DMI_MATCH(DMI_BOARD_VENDOR, "Gigabyte Technology Co., Ltd."),
> + DMI_MATCH(DMI_BOARD_NAME, "X299 DESIGNARE EX-CF"),
> + },
> + },
> + { }
> +};
> +
> /**
> * pci_bridge_d3_possible - Is it possible to put the bridge into D3
> * @bridge: Bridge to check
> @@ -2546,6 +2563,9 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge)
> if (bridge->is_hotplug_bridge)
> return false;
>
> + if (dmi_check_system(bridge_d3_blacklist))
> + return false;
> +
> /*
> * It should be safe to put PCIe ports from 2015 or newer
> * to D3.
> --
> 2.19.2
>
