[+ Vladimir and Alex]
Am 2020-12-09 13:36, schrieb Bjorn Helgaas:
On Tue, Dec 08, 2020 at 04:41:50PM +0100, Michael Walle wrote:
>On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
>> Unify ECAM-related constants into a single set of standard constants
>> defining memory address shift values for the byte-level address that can
>> be used when accessing the PCI Express Configuration Space, and then
>> move native PCI Express controller drivers to use newly introduced
>> definitions retiring any driver-specific ones.
>>
>> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
>> PCI Express specification (see PCI Express Base Specification, Revision
>> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
>> implement it the same way.
>>
>> [...]
>
>Applied to pci/ecam, thanks!
>
>[1/5] PCI: Unify ECAM constants in native PCI Express drivers
> https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
Patch 1/5 breaks LS1028A boards:
[..]
[ 1.144426] pci-host-generic 1f0000000.pcie: host bridge
/soc/pcie@1f0000000 ranges:
[ 1.152276] pci-host-generic 1f0000000.pcie: MEM
0x01f8000000..0x01f815ffff -> 0x0000000000
[ 1.161161] pci-host-generic 1f0000000.pcie: MEM
0x01f8160000..0x01f81cffff -> 0x0000000000
[ 1.170043] pci-host-generic 1f0000000.pcie: MEM
0x01f81d0000..0x01f81effff -> 0x0000000000
[ 1.178924] pci-host-generic 1f0000000.pcie: MEM
0x01f81f0000..0x01f820ffff -> 0x0000000000
[ 1.187805] pci-host-generic 1f0000000.pcie: MEM
0x01f8210000..0x01f822ffff -> 0x0000000000
[ 1.196686] pci-host-generic 1f0000000.pcie: MEM
0x01f8230000..0x01f824ffff -> 0x0000000000
[ 1.205562] pci-host-generic 1f0000000.pcie: MEM
0x01fc000000..0x01fc3fffff -> 0x0000000000
Can you attach your DT?
That would be the following:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28-var3-ads2.dts
You'll find the PCI devices/bridge in:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
The fact that all these windows map to PCI
bus address 0 looks broken. Prior to patch 1/5, do the devices below
this bridge actually work?
Yes, these should be the onboard network controller and ethernet
switch.
Looks like you're using the pci-host-generic driver; which of the
.compatible strings (pci-host-cam-generic, pci-host-ecam-generic,
marvell,armada8k-pcie-ecam, etc) are you using? (I think that's in
the DT as well.)
compatible = "pci-host-ecam-generic";
[ 1.214465] pci-host-generic 1f0000000.pcie: ECAM at [mem
0x1f0000000-0x1f00fffff] for [bus 00]
[ 1.223318] pci-host-generic 1f0000000.pcie: PCI host bridge to bus
0000:00
[ 1.230350] pci_bus 0000:00: root bus resource [bus 00]
[ 1.235625] pci_bus 0000:00: root bus resource [mem
0x1f8000000-0x1f815ffff] (bus address [0x00000000-0x0015ffff])
[ 1.246077] pci_bus 0000:00: root bus resource [mem
0x1f8160000-0x1f81cffff pref] (bus address [0x00000000-0x0006ffff])
[ 1.256969] pci_bus 0000:00: root bus resource [mem
0x1f81d0000-0x1f81effff] (bus address [0x00000000-0x0001ffff])
[ 1.267427] pci_bus 0000:00: root bus resource [mem
0x1f81f0000-0x1f820ffff pref] (bus address [0x00000000-0x0001ffff])
[ 1.278326] pci_bus 0000:00: root bus resource [mem
0x1f8210000-0x1f822ffff] (bus address [0x00000000-0x0001ffff])
[ 1.288779] pci_bus 0000:00: root bus resource [mem
0x1f8230000-0x1f824ffff pref] (bus address [0x00000000-0x0001ffff])
[ 1.299669] pci_bus 0000:00: root bus resource [mem
0x1fc000000-0x1fc3fffff] (bus address [0x00000000-0x003fffff])
[ 1.310138] pci 0000:00:00.0: [1957:e100] type 00 class 0x020001
[ 1.316234] pci 0000:00:00.0: BAR 0: [mem 0x1f8000000-0x1f803ffff
64bit] (from Enhanced Allocation, properties 0x0)
[ 1.326776] pci 0000:00:00.0: BAR 2: [mem 0x1f8160000-0x1f816ffff
64bit pref] (from Enhanced Allocation, properties 0x1)
[ 1.337759] pci 0000:00:00.0: VF BAR 0: [mem
0x1f81d0000-0x1f81dffff 64bit] (from Enhanced Allocation, properties
0x4)
[ 1.348563] pci 0000:00:00.0: VF BAR 2: [mem
0x1f81f0000-0x1f81fffff 64bit pref] (from Enhanced Allocation,
properties 0x3)
[ 1.359821] pci 0000:00:00.0: PME# supported from D0 D3hot
[ 1.365368] pci 0000:00:00.0: VF(n) BAR0 space: [mem
0x1f81d0000-0x1f81effff 64bit] (contains BAR0 for 2 VFs)
[ 1.375381] pci 0000:00:00.0: VF(n) BAR2 space: [mem
0x1f81f0000-0x1f820ffff 64bit pref] (contains BAR2 for 2 VFs)
[ 1.385983] Unable to handle kernel paging request at virtual
address ffff800012132000
If ffff800012132000 were an actual ECAM address, we would expect the
low 20 bits to contain the device number, function number, and
config register offset, i.e.,
dev (0xffff800012132000 >> 15) & 0x01f = 0x6
fn (0xffff800012132000 >> 12) & 0x007 = 0x2
reg (0xffff800012132000) & 0xfff = 0
but that's non-sensical since we probe for devices in order. So maybe
this is a bad pointer somewhere else. I looked at pci_ecam_map_bus()
but didn't see an obvious problem. Maybe we could brute-force debug
this by adding some printks there.
I'm not that deep into PCI, but if you could tell me what to add where,
I'm happy to help ;)
[ 1.393972] Mem abort info:
[ 1.396783] ESR = 0x96000007
[ 1.399859] EC = 0x25: DABT (current EL), IL = 32 bits
[ 1.405215] SET = 0, FnV = 0
[ 1.408290] EA = 0, S1PTW = 0
[ 1.411453] Data abort info:
[ 1.414352] ISV = 0, ISS = 0x00000007
[ 1.418216] CM = 0, WnR = 0
[ 1.421205] swapper pgtable: 4k pages, 48-bit VAs,
pgdp=000000008369c000
[ 1.427966] [ffff800012132000] pgd=00000020fffff003,
p4d=00000020fffff003, pud=00000020ffffe003, pmd=00000020ffffa003,
pte=0000000000000000
[ 1.440618] Internal error: Oops: 96000007 [#1] PREEMPT SMP
[ 1.446239] Modules linked in:
[ 1.449320] CPU: 1 PID: 1 Comm: swapper/0 Not tainted
5.10.0-rc3-00101-g2f378db5c89 #191
[ 1.457484] Hardware name: Kontron SMARC-sAL28 (Single PHY) on
SMARC Eval 2.0 carrier (DT)
[ 1.465827] pstate: 20000085 (nzCv daIf -PAN -UAO -TCO BTYPE=--)
[ 1.471892] pc : pci_generic_config_read+0x38/0xe0
[ 1.476723] lr : pci_generic_config_read+0x24/0xe0
[ 1.481553] sp : ffff80001211b920
[ 1.484891] x29: ffff80001211b920 x28: 0000000000000000
[ 1.490252] x27: ffff8000116a04bc x26: 0000000000000000
[ 1.495612] x25: 0000000000000001 x24: ffff80001211ba54
[ 1.500972] x23: ffff0020009c3800 x22: 0000000000000000
[ 1.506332] x21: 0000000000000087 x20: ffff80001211b994
[ 1.511692] x19: 0000000000000004 x18: 0000000000000000
[ 1.517052] x17: 0000000000000000 x16: 00000000d5edfbc1
[ 1.522412] x15: ffffffffffffffff x14: ffff800011cf9948
[ 1.527772] x13: ffff002000305a1c x12: 0000000000000030
[ 1.533132] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f
[ 1.538491] x9 : 2c6b7173626d686f x8 : 000000000000ea60
[ 1.543851] x7 : ffff80001211ba54 x6 : 0000000000000000
[ 1.549211] x5 : 0000000000000000 x4 : ffff800012131000
[ 1.554570] x3 : 0000000000000000 x2 : 0000000000000000
[ 1.559930] x1 : 0000000000001000 x0 : ffff800012132000
[ 1.565290] Call trace:
[ 1.567752] pci_generic_config_read+0x38/0xe0
[ 1.572233] pci_bus_read_config_dword+0x84/0xd8
[ 1.576890] pci_bus_generic_read_dev_vendor_id+0x34/0x1b0
[ 1.582423] pci_bus_read_dev_vendor_id+0x4c/0x70
[ 1.587167] pci_scan_single_device+0x84/0xe0
[ 1.591559] pci_scan_slot+0x6c/0x120
[ 1.595250] pci_scan_child_bus_extend+0x54/0x298
[ 1.599994] pci_scan_root_bus_bridge+0xd4/0xf0
[ 1.604562] pci_host_probe+0x18/0xb0
[ 1.608254] pci_host_common_probe+0x13c/0x1a0
[ 1.612735] platform_drv_probe+0x54/0xa8
[ 1.616777] really_probe+0xe4/0x3b8
[ 1.620380] driver_probe_device+0x58/0xb8
[ 1.624509] device_driver_attach+0x74/0x80
[ 1.628725] __driver_attach+0x58/0xe0
[ 1.632503] bus_for_each_dev+0x74/0xc8
[ 1.636369] driver_attach+0x24/0x30
[ 1.639972] bus_add_driver+0x18c/0x1f0
[ 1.643838] driver_register+0x64/0x120
[ 1.647704] __platform_driver_register+0x48/0x58
[ 1.652449] gen_pci_driver_init+0x1c/0x28
[ 1.656580] do_one_initcall+0x4c/0x2c0
[ 1.660447] kernel_init_freeable+0x1e4/0x250
[ 1.664840] kernel_init+0x14/0x118
[ 1.668355] ret_from_fork+0x10/0x34
[ 1.671961] Code: 7100067f 540001c0 71000a7f 54000300 (b9400001)
[ 1.678114] ---[ end trace 0aca1b048661e8b3 ]---
[ 1.682770] note: swapper/0[1] exited with preempt_count 1
[ 1.688305] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[ 1.696031] SMP: stopping secondary CPUs
[ 1.699989] Kernel Offset: disabled
[ 1.703503] CPU features: 0x0240022,61006008
[ 1.707806] Memory Limit: none
[ 1.710884] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b ]---
There is a LS1028A eval board in kernelci here:
https://lavalab.nxp.com/scheduler/job/170566
I actually have this board which also have a LS1028A SoC:
https://lavalab.kontron.com/scheduler/job/1771
But in the latter you won't see much because earlycon isn't active.
[I'm
about to fix that.]
By reverting patch 1/5, the board will work again.
-michael
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
-michael
