On Thu, 2009-10-29 at 12:28 -0700, Eric W. Biederman wrote: > Bjorn Helgaas <bjorn.helgaas@xxxxxx> writes: > > > On Thu, 2009-10-29 at 08:13 -0700, Eric W. Biederman wrote: > >> Bjorn Helgaas <bjorn.helgaas@xxxxxx> writes: > >> > >> > On Thu, 2009-10-29 at 01:16 -0700, Eric W. Biederman wrote: > >> >> Yinghai Lu <yinghai@xxxxxxxxxx> writes: > >> >> > > >> >> > after closing look up the code, it looks it will not break your setup. > >> >> > > >> >> > 1. before the patches: > >> >> > a. when master card is inserted, all bridge in that card will get assigned with min_size > >> >> > b. when new cards is inserted to those slots in master card, will get assigned in the bridge size. > >> >> > > >> >> > 2. after the patches: v5 > >> >> > a. booted up, all leaf bridge mmio get clearred. > >> >> > b. when master card is inserted, all bridge in that card will get assigned with min_size, and master bridge will be sum of them > >> >> > c. when new cards is inserted to those slots in master card, will get assigned in the bridge size. > >> >> > > >> >> > can you check those two patches in your setup to verify it? > >> >> > >> >> I have a much simpler case I will break, as I tried something similar by accident. > >> >> > >> >> AMD cpu MCP55 with one pcie port setup as hotplug. > >> >> The system only has 2GB of RAM. So plenty of space for pcie devices. > >> >> > >> >> If the firmware assigns nothing and linux at boot time assigns the pci mmio space: > >> >> Reads from the bar of the hotplugged device work > >> >> Writes to the bar of the hotplugged device, cause further writes to go to lala land. > >> >> > >> >> So I had to have the firmware make the assignment, because only it knows the > >> >> details of the hidden AMD bar registers for each hypertransport chain etc. > >> > > >> > Do you mean you had to have firmware program a hot-added device, or just > >> > that firmware had to program the apertures of the root port that was > >> > present at boot, even though it had no devices below it? > >> > >> Firmware had to program the apertures of the root port that was present > >> at boot, even though it had no devices below it. > >> > >> > Firmware normally supplies ACPI _CRS information that tells us how it > >> > programmed the host bridge windows. On x86, Linux normally ignores that > >> > and just assumes a range based on memory size. If we paid attention to > >> > it (as with "pci=use_crs"), it's likely that we could do a better job of > >> > doing this setup. > >> > > >> > Or, of course, we could add a Linux driver that knows about "the hidden > >> > AMD bar registers." But I think that should be a last resort, for when > >> > firmware supplied incorrect _CRS information. > >> > >> In this case there was no ACPI, and even if there was correct _CRS information > >> it would have said only those addresses routed to bars/apertures on the > >> root bridge was routed to the MCP55. So while it looked like we had gobs > >> of unallocated space we could use. In practice we did not. > > > > I know this is a hypothetical case since you don't have ACPI, but I'm > > curious about this. > > > > I assume the magic AMD BARs only affect the host bridge, and that the > > downstream root ports look like standard PCI-to-PCI bridges. If that's > > the case, and if we have correct descriptions of the host bridge > > apertures, Linux should theoretically be able to do as well as firmware. > > > > But you seem to be suggesting that even with a correct host bridge > > description, there's space that *looks* available but is not. I don't > > understand how this can be. > > What I meant was simply that not all of the non-memory space was > routed down the hypertransport chain to the mcp55. If you have an > accurate description of that you should be fine. OK, yep, that makes perfect sense. That's a good example of why I believe we should start paying attention to the root bridge _CRS, because that's exactly what it would tell us. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
