On Fri, May 15, 2015 at 01:04:22PM -0700, Guenter Roeck wrote: > Hi Bjorn, > > On Fri, May 15, 2015 at 02:25:34PM -0500, Bjorn Helgaas wrote: > > [+cc Yinghai] > > > > Hi Guenter, > > > > On Fri, May 15, 2015 at 12:28 PM, Guenter Roeck <groeck@xxxxxxxxxxx> wrote: > > > Hi all, > > > > > > ever since 3.18(ish), we see lots of pcie io range allocation error messages > > > when booting our systems. > > > > > > pcieport 0000:02:00.0: res[7]=[io 0x1000-0x0fff] get_res_add_size add_size 1000 > > > pcieport 0000:02:00.0: BAR 7: no space for [io size 0x1000] > > > pcieport 0000:02:00.0: BAR 7: failed to assign [io size 0x1000] > > > > > > This is repeated for each port. > > > > > > What we atually _want_ is no IO address range, but so far I have not been able to > > > tell this to the code (or to the bridge). > > > > What information does the core have that we could use to figure out > > that you want no I/O space? For example, does the host bridge have no > > I/O apertures? Or are there no devices below 02:00.0 that require I/O > > space? If you can post a complete dmesg log, that should contain this > > information. > > > We know that we don't need IO space behind the bridge. > > Question would be what information the core would need. I have tried > to pre-program PCI_IO_BASE and PCI_IO_LIMIT on the bridge ports, > but that doesn't seem to be the right solution (and it was just a > wild shot anyway ;-). The core tries to assign address space before drivers start claiming devices because it's very difficult to move devices after drivers are attached. To avoid assigning I/O space to a bridge, the core would have to know that (1) no devices below the bridge have I/O BARs, and (2) there's no hotplug slot where a new device could be added below the bridge. In your specific case, I suspect that you have hotplug bridges, but you *know* exactly what devices can be "hot-plugged" and you know they don't need I/O space. But there's no good way to tell that to the PCI core. > > Is the problem that some device doesn't work, or is it just extra > > annoying messages. We should fix something in either case, of course, > > but it's more urgent if a device used to work but no longer does. > > > Just the extra annoying messages. The messages below are just the beginning; > there are many more when the actual devices behind the bridge ports come > online. This sounds like an interesting topology. A device coming online later is a hotplug event from the core's point of view. Since the core can't tell what a hot-added device is going to be, it allocates a default amount of space (pci_hotplug_io_size) to accomodate it. You can influence that with the "pci=hpiosize=" boot parameter. Maybe "pci=hpiosize=0" would help in your case, but we'd still have the problem that the messages don't make sense to the average user (including me). fsl-pci fff70a000.pcie: PCI host bridge to bus 0000:00 pci_bus 0000:00: root bus resource [io 0x0000-0xffff] pci 0000:00:00.0: PCI bridge to [bus 01-ff] # Root Port pci 0000:00:00.0: bridge window [io 0x0000-0x0fff] The host bridge does have an I/O aperture, and it looks like firmware set up the 00:00.0 bridge with a small piece of that. That's sort of a waste: supposedly we have 64K of I/O space available on bus 00, and 4K of that is routed to [bus 01-ff]. The other 60K could be used by other devices on bus 00, but it doesn't look like there *are* any, so it is wasted. pci 0000:01:00.0: PCI bridge to [bus 02-ff] # Upstream Port pci 0000:02:00.0: PCI bridge to [bus 40-4f] # Downstream Port 0x0 ... pci 0000:02:0d.0: PCI bridge to [bus a0-af] # Downstream Port 0xa pci 0000:02:00.0: bridge window [io 0x1000-0x0fff] to [bus 40-4f] add_size 1000 ... pci 0000:02:0d.0: bridge window [io 0x1000-0x0fff] to [bus a0-af] add_size 1000 I'm guessing these Downstream Ports all have "is_hotplug_bridge" set because they have the "Hot-Plug Capable" bit set in their Slot Capabilities register. We're trying to allocate 0x1000 I/O ports for each, and there are 0xb of them, so we'd need 0xb000 I/O ports at the 01:00.0 bridge: pci 0000:01:00.0: bridge window [io 0x1000-0x0fff] to [bus 02-ff] add_size b000 pci_hotplug_io_size defaults to 256, but standard PCI bridges don't support I/O windows smaller than 4K, so they all got rounded up to 4K (0x1000). I would love it if somebody would clean up those messages and make them all consistent. There's useful information there, so I'm not sure we want to get rid of them, but I think there's some redundancy we might be able to fix. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
