On Fri, Dec 08, 2023 at 04:29:42PM -0600, Mario Limonciello wrote: > On 12/8/2023 16:24, Bjorn Helgaas wrote: > > On Wed, Aug 16, 2023 at 10:49:23AM +0530, Sanath S wrote: > > > In the case of Thunderbolt, it contains a PCIe switch and one or > > > more hotplug-capable PCIe downstream ports where the daisy chain > > > can be extended. > > > > > > Currently when a Thunderbolt Dock is plugged in during S5/Reboot, > > > System BIOS allocates a very minimal number of buses for bridges and > > > hot-plug capable PCIe downstream ports to enumerate the dock during > > > boot. Because of this, we run out of bus space pretty quickly when > > > more PCIe devices are attached to hotplug downstream ports in order > > > to extend the chain. > > > > > > Before: > > > +-04.0 > > > +-04.1-[63-c1]----00.0-[64-69]--+-00.0-[65]-- > > > | +-01.0-[66]-- > > > | +-02.0-[67]-- > > > | +-03.0-[68]-- > > > | \-04.0-[69]-- > > > +-08.0 > > > > Looks like a clear issue here because there's no other use for > > buses 70-c1. But what would happen if there were more hotplug-capable > > downstream ports, e.g., assume one at 08.1 leading to [bus c2-c7]? > > > > The 04.1 bridge has a lot of space, but 08.1 has very little. With > > this patch, would we distribute it more evenly across 04.1 and 08.1? > > If not, I think we'll just have the same problem when somebody plugs > > in a similar hierarchy at 08.1. > > > > > In case of a thunderbolt capable bridge, reconfigure the buses allocated > > > by BIOS to the maximum available buses. So that the hot-plug bridges gets > > > maximum buses and chain can be extended to accommodate more PCIe devices. > > > This fix is necessary for all the PCIe downstream ports where the daisy > > > chain can be extended. > > > > > > After: > > > +-04.0 > > > +-04.1-[63-c1]----00.0-[64-c1]--+-00.0-[65]-- > > > | +-01.0-[66-84]-- > > > | +-02.0-[85-a3]-- > > > | +-03.0-[a4-c0]-- > > > | \-04.0-[c1]-- > > > +-08.0 > > > > This doesn't look like anything specific to Thunderbolt; it's just > > that we don't do a good job of reassigning bus numbers in general, > > right? We shouldn't just punt and say "BIOS should have done > > something" because not all machines *have* BIOS, and the OS can > > reconfigure bus numbers as needed. The patch certainly isn't > > Thunderbolt-specific. > > From the discussions Sanath and I have been in related to this issue > the BIOS is pretty static with it's initialization under the > presumption that the OS will rebalance things if necessary. > ... > For this particular issue it's being approached a different way. > > Windows never rebalances things but doesn't suffer from this issue. > That's because Windows actually does a "Downstream port reset" when > it encounters a USB4 router. > > Sanath posted a quirk that aligned this behavior when encountering > an AMD USB4 router, but as part of the discussion I suggested that > we do it for everyone. > > https://lore.kernel.org/linux-usb/20231123065739.GC1074920@xxxxxxxxxxxxxxxxxx/ > > So Sanath has a new patch that does this that is under testing right > now and will be posted soon. Hmm, ok. I don't know what a "downstream port reset" does or how it resolves the bus number allocation issue, but I'm happy if you have a fix that doesn't need PCI core changes. Bjorn
