On Mon, Feb 27, 2017 at 07:25:58PM -0500, Joshua Kinard wrote: > On 02/27/2017 11:36, Bjorn Helgaas wrote: > > On Sat, Feb 25, 2017 at 04:34:12AM -0500, Joshua Kinard wrote: > >> On 02/24/2017 13:38, Bjorn Helgaas wrote: > >>> On Fri, Feb 24, 2017 at 03:50:26AM -0500, Joshua Kinard wrote: > So the logic used to read the vendor/dev IDs can probably be > extended to poke the BARs via the ~0 trick OR read out the > pre-defined mappings from ARCS firmware, then use that info to size > out the BRIDGE windows into Crosstalk space and then set up our > IORESOURCE_MEM and IORESOURCE_IO structs with the right information > to pass into register_pci_controller(). As you probably know, you can determine the size of a BAR regardless of whether it has been assigned. So it doesn't matter whether ARCS has already assigned anything. See __pci_read_base(). > I dunno, I'll probably try reading the ARCS mappings idea first, as > that seems easier. I won't have a lot of time to play with things > in March, so getting something to work, even if it is sub-optimal, > is better than nothing working at all. The simplest possible thing is to enable some pre-defined set of windows and accept that this may lead to configuration restrictions, e.g., a device with large BARs may work only in certain slots. > Each window, at least on Octane, lives at distinct addresses in > Crosstalk space. A specific address within each window can point to > the same device on a subordinate PCI bus, but I think that will be > transparent to the Linux PCI core. My thinking is, if I setup > IORESOURCE_MEM structs for each window for each BRIDGE widget, then > the PCI code will check each struct to see which one contains the > address range that the PCI device's BAR wants. > > E.g., Using widget 0xd as an example for readability, if I know my > three windows in Crosstalk space are: > Small: 0x0000_1d00_0000 - 0x0000_1dff_ffff > Medium: 0x000e_8000_0000 - 0x000e_ffff_ffff > Large: 0x00d0_0000_0000 - 0x00df_ffff_ffff > > And for that specific BRIDGE, if I pass those three ranges as > IORESOURCE_MEM structs, then shouldn't the PCI core, if it is told > that a specific devices BAR wants range 1d200000 to 1d203fff, select > the small window mapping? E,g., is it going to try to find the > smallest possible window to fit first? Or will it instead try to > match using the large window? No, when matching a BAR value to a host bridge window, the PCI core will definitely not look for the smallest (or largest) window that contains the BAR. The core reads the BAR value (a bus address), then searches the host bridge windows for one that maps to a region that contains the bus address. This happens in pcibios_bus_to_resource(), which is called by __pci_read_base(). The search order is undefined because the core assumes the windows do not overlap in PCI bus address space. It can't deal with two windows that map to the same PCI space because when we allocate space when assigning a BAR, we search for available CPU address space, not for available PCI bus address space. If window A and window B both map to PCI bus address X, we may assign space from window A to PCI BAR 1 and space from window B to PCI BAR 2. Then we have two BARs at the same PCI bus address, which will cause conflicts. I don't think we currently check in the PCI core for host bridge windows that map to overlapping PCI bus space. Maybe we should, and just reject anything that overlaps. Bjorn
