On Wed, Nov 04, 2009 at 04:25:41PM -0800, Ira W. Snyder wrote: > On Wed, Nov 04, 2009 at 06:18:19PM -0600, Bjorn Helgaas wrote: > > On Wednesday 04 November 2009 04:59:29 pm Ira W. Snyder wrote: > > > On Wed, Nov 04, 2009 at 05:15:52PM -0600, Bjorn Helgaas wrote: > > > > On Wednesday 04 November 2009 10:52:50 am Ira W. Snyder wrote: > > > > > ... > > > > > All six scenarios work on the Force computer. It seems to re-assign the > > > > > same PCI BAR addresses each time. > > > > > > > > > > All six scenarios fail on the Trenton computer. It always re-assigns the > > > > > PCI BAR addresses starting at 0x40000000, which is just above the top of > > > > > physical memory. There is 1GB of RAM in the system. > > > > > > > > I assume that if the card is present at boot, it works until you > > > > try the hotplug operations. > > > > > > > > > On both the Trenton and Force computers, the BIOS assigns the PCI BAR > > > > > addresses towards the top of memory. Both assign addresses above > > > > > 0xF0000000. > > > > > > > > > > After running a hotplug on the Trenton computer, I have used a PCI logic > > > > > analyzer to confirm that no transactions accessing the PCI BAR's of the > > > > > hotplugged card make it onto the PCI bus. PCI configuration space still > > > > > works. > > > > > > > > If the bridge isn't forwarding transactions to the PCI bus, my guess > > > > is that we put the BAR the wrong place, e.g., maybe we didn't place > > > > it in a host bridge aperture. > > > > > > > > This is perfect timing ... I'm trying to improve the Linux PCI resource > > > > messages to help debug situations like this. If it's convenient for you, > > > > I would love to see dmesg logs from both boards using the current PCI > > > > linux-next kernel. > > > > > > > > > > I've attached logs from both boards. Note how running "mapper" after > > > hotplug works fine on the Force, but returns all 0xff's on the Trenton. > > > That's where I hooked up a PCI logic analyzer and verified that > > > *nothing* ever makes it onto the bus. > > > > Hmm, no ACPI on either board :-) (I assume you expected that.) > > > > Yep, no ACPI. It doesn't work, or is missing from the BIOS, or something > like that. ACPI is on in the kernel config. > > > In that case, we can only guess the host bridge apertures based on > > the memory size, since we don't have host bridge drivers that know > > how to read them out of the hardware. > > > > You mentioned using "memmap=" to change the aperture we allocate > > from. Can you tweak that to make the after-hotplug mappings be > > closer to the original BIOS ones? I know you tried that already; > > I guess I'm just wondering if the magic lower boundary is something > > like 0xfb000000 rather than 0xf0000000. > > > > I can try that tomorrow. I'll keep bumping it up 16MB at a time and see > what happens. > Ok, I just ran these tests. Using memmap= to reserve memory and force Linux to place allocations higher didn't work at all. I started at 0xf8000000 and moved up from there. Even after getting so the BARs were re-assigned to higher addresses than the original BIOS addresses didn't help. Reading the BAR still produces all 0xff's. Here are some logs from my last run, which show the BIOS and Linux assigned addresses: [ 0.372360] pci 0000:01:0b.0: reg 10: [mem 0xfeb00000-0xfebfffff] [ 0.376051] pci 0000:01:0b.0: reg 14: [mem 0xfbfff000-0xfbffffff pref] [ 0.380029] pci 0000:01:0b.0: reg 18: [mem 0xfbe00000-0xfbefffff 64bit pref] [ 0.384028] pci 0000:01:0b.0: reg 20: [mem 0xfbd00000-0xfbdfffff 64bit pref] [ 146.650835] pci 0000:01:0b.0: reg 10: [mem 0xfeb00000-0xfebfffff] [ 146.656982] pci 0000:01:0b.0: reg 14: [mem 0xfbfff000-0xfbffffff pref] [ 146.663533] pci 0000:01:0b.0: reg 18: [mem 0xfbe00000-0xfbefffff 64bit pref] [ 146.670615] pci 0000:01:0b.0: reg 20: [mem 0xfbd00000-0xfbdfffff 64bit pref] [ 146.677856] pci 0000:01:0b.0: BAR 0: assigned [mem 0xfef00000-0xfeffffff] [ 146.684655] pci 0000:01:0b.0: BAR 0: set to [mem 0xfef00000-0xfeffffff] (PCI address [0xfef00000-0xfeffffff] [ 146.694487] pci 0000:01:0b.0: BAR 2: assigned [mem 0xff000000-0xff0fffff 64bit pref] [ 146.702247] pci 0000:01:0b.0: BAR 2: set to [mem 0xff000000-0xff0fffff 64bit pref] (PCI address [0xff000000-0xff0fffff] [ 146.713031] pci 0000:01:0b.0: BAR 4: assigned [mem 0xff100000-0xff1fffff 64bit pref] [ 146.720832] pci 0000:01:0b.0: BAR 4: set to [mem 0xff100000-0xff1fffff 64bit pref] (PCI address [0xff100000-0xff1fffff] [ 146.731639] pci 0000:01:0b.0: BAR 1: assigned [mem 0xfee01000-0xfee01fff pref] [ 146.738871] pci 0000:01:0b.0: BAR 1: set to [mem 0xfee01000-0xfee01fff pref] (PCI address [0xfee01000-0xfee01fff] Any more ideas? Thanks, Ira -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
