Hi Bjorn, Thanks for your help. On Fri, Mar 6, 2020 at 9:47 PM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote: > > [+cc Thomas, Jason, Nicholas, Ben] > > On Fri, Mar 06, 2020 at 02:32:59PM +0000, Luís Mendes wrote: > > Hi, > > > > I'm trying to use Google/Coral TPU Edge modules for a project, on > > arm64 and armhf, but BAR0 doesn't get assigned during the enumeration > > of PCIe devices and consequently pci_enable_device(...) fails on BAR0 > > resource with value -22 (EINVAL) (resource has null parent) when > > loading gasket/apex driver. > > > > I'm also trying to adapt gasket/apex to run on armhf, but anyhow that > > is not the root cause for this issue. > > > > Relevant Log extracts follow in attachment. > > Hi Luís, > > Thanks for the report, and sorry for the problem you're tripping over. > I cc'd a few folks who might be interested. > > > [ 6.983880] mvebu-pcie soc:pcie: /soc/pcie/pcie@1,0: reset gpio is active low > > [ 6.993528] hub 4-1:1.0: 4 ports detected > > [ 6.993749] mvebu-pcie soc:pcie: /soc/pcie/pcie@2,0: reset gpio is active low > > [ 7.106741] sdb: sdb1 > > [ 7.109826] sd 2:0:0:0: [sdb] Attached SCSI removable disk > > [ 7.242916] mvebu-pcie soc:pcie: PCI host bridge to bus 0000:00 > > [ 7.248854] pci_bus 0000:00: root bus resource [bus 00-ff] > > [ 7.254370] pci_bus 0000:00: root bus resource [mem 0xd0000000-0xefffffff] > > [ 7.261267] pci_bus 0000:00: root bus resource [io 0x1000-0xeffff] > > [ 7.267621] pci 0000:00:01.0: [11ab:6828] type 01 class 0x060400 > > [ 7.273662] pci 0000:00:01.0: reg 0x38: [mem 0x00000000-0x000007ff pref] > > [ 7.293971] PCI: bus0: Fast back to back transfers disabled > > [ 7.299558] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring > > [ 7.315694] pci 0000:01:00.0: [1ac1:089a] type 00 class 0x0000ff > > [ 7.321749] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit pref] > > [ 7.322814] usb 4-1.1: new high-speed USB device number 3 using xhci-hcd > > [ 7.329004] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x000fffff 64bit pref] > > [ 7.343111] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:01.0 (capable of 4.000 Gb/s with 5 GT/s x1 link) > > [ 7.383442] PCI: bus1: Fast back to back transfers disabled > > [ 7.389031] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 > > [ 7.495604] pci 0000:00:02.0: ASPM: current common clock configuration is broken, reconfiguring > > [ 7.552513] pci 0000:00:01.0: BAR 8: assigned [mem 0xe8000000-0xe81fffff] > > [ 7.565611] pci 0000:00:01.0: BAR 6: assigned [mem 0xe8200000-0xe82007ff pref] > > [ 7.580096] pci 0000:00:01.0: PCI bridge to [bus 01] > > [ 7.585079] pci 0000:00:01.0: bridge window [mem 0xe8000000-0xe81fffff] > > [ 7.653228] pcieport 0000:00:01.0: enabling device (0140 -> 0142) > > > > > > [ 11.188025] gasket: module is from the staging directory, the quality is unknown, you have been warned. > > [ 11.217048] apex: module is from the staging directory, the quality is unknown, you have been warned. > > [ 11.217926] apex 0000:01:00.0: can't enable device: BAR 0 [mem 0x00000000-0x00003fff 64bit pref] not claimed > > [ 11.227825] apex 0000:01:00.0: error enabling PCI device > > It looks like we did assign space for the bridge window leading to > 01:00.0, but failed to assign space to the 01:00.0 BAR itself. > > I don't know offhand why that would be. Can you put the entire dmesg > log somewhere we can see? That will tell us what kernel you're using > and possibly other useful things. Sure, complete dmesg is available at: https://pastebin.ubuntu.com/p/qnzJ56kM7k/ This is a custom built machine based on the Armada 388 armhf SoC, that I am using, but I've also tried an arm64 machine from Toradex, an Apalis IMX8QM with the Apalis Eval board, producing similar results with different kernels and also different ARM architectures. > > Does the same problem happen with other devices, or does it only > happen with the gasket/apex combination? There shouldn't be anything > device-specific in the PCI core resource assignment code. This issue seems to happen only with the Coral Edge TPU device, but it happens independently of whether the gasket/apex driver module is loaded or not. The BAR 0 of the Coral device is not assigned either way. Luís
