On 3/13/24 13:48, Daniel Bryant wrote:
I'm using the of_pci_make_dev_node() framework to dynamically create
nodes for the host bridge and attached device, then applying an overlay to
create a simple-bus and children on the endpoint device. The initial problem
is that none of the children create any memory resources. This is on
a DT-based system, aarch64 hardware.
The overall bus hierarchy is this:
PCIe root port controller -> host bridge -> device -> simple-bus ->
[subdevices].
1. Root port controller is defined by static DT blob.
2. bridge and device of_nodes are created by of_pci_make_dev_node()
3. simple-bus and subdevices are added from a single overlay.
The end result is very similar to the test case in
drivers/of/unittest-data/tests-address.dtsi. I've traced the failure
for this hardware to of_translate_address() always returning
OF_BAD_ADDR for any of the "reg" properties of a subdevice.The
critical difference between the working unit test and the hardware DT
is that the hardware PCIe root port controller has a ranges property
that's not just an identity mapping.
arch/arm64/boot/dts/nvidia/tegra234.dtsi contains the following
definition, note specifically the second ranges entry:
pcie@14160000 {
compatible = "nvidia,tegra234-pcie";
device_type = "pci";
// ...
ranges = <0x43000000 0x21 0x40000000 0x21 0x40000000 0x2
0xe8000000>, /* prefetchable memory (11904 MB) */
<0x02000000 0x0 0x40000000 0x24 0x28000000 0x0
0x08000000>, /* non-prefetchable memory (128 MB) */
<0x01000000 0x0 0x36100000 0x00 0x36100000 0x0
0x00100000>; /* downstream I/O (1 MB) */
};
Then when the child bridge is created, a of_node is created with the
following properties:
pci@0,0 {
compatible = "pci10de,229c\0pciclass,060400\0pciclass,0604";
#address-cells = <0x03>;
#size-cells = <0x02>;
bus-range = <0x01 0xff>;
device_type = "pci";
ranges = <0x82000000 0x24 0x28000000 0x82000000 0x24
0x28000000 0x00 0x200000>;
};
This ranges entry is a simple passthrough, but it's of the addresses
already translated to the CPU address space, and not the bus address.
The format of 'range' entry is <child addr> <parent addr> <size>.
the parent address is (0x82000000 0x24 0x28000000) which is a pci
address (not CPU address).
This address should match one of the range entry of its parent.
0x82000000 indicates 32-bit iomem. But the offset is 64-bit. (0x24
0x28000000) which looks odd.
I would suggest to look into lspci -v output as well and debug in
of_bus_pci_map() to see why it does not match any entry.
Thanks,
Lizhi
This will cause of_translate_address() to fail when it reaches
pcie@14160000 and doesn't match.
As a caveat, I'm currently seeing this on hardware on a downstream
vendor kernel. I'm still working to reproduce on mainline 6.8 (via
qemu, probably?), but I don't yet see anything that would stop this
from being any different there.
Dan
