On Thu, Mar 13, 2025 at 06:56:17PM -0400, Frank Li wrote:
> On Thu, Mar 13, 2025 at 05:04:50PM -0500, Bjorn Helgaas wrote:
> > On Thu, Mar 13, 2025 at 11:38:42AM -0400, Frank Li wrote:
> > > The 'ranges' property at PCI controller parent bus can indicate address
> > > translation information. Most system's bus fabric use 1:1 map between input
> > > and output address. but some hardware like i.MX8QXP doesn't use 1:1 map.
> >
> > I think you've used reg["addr_space"] to get the offset for Endpoints
> > forever.
>
> Yes, it still need ranges informaiton at parent bus.
>
> bus@000
> {
> ranges = <...>; [1] /* still need this */
> pcie {
> ranges = <...>;[2]
> };
> pcie-ep {};
> }
>
> >
> > I just noticed that through v9, you used 'ranges' to get the offset
> > for the Root Complex (with "Add parent_bus_offset to resource_entry"),
> > and I think v10 switched to use reg["config"] instead.
> >
> > I think I originally proposed the idea of "Add parent_bus_offset to
> > resource_entry" patch, but I think it turned out to be kind of an ugly
> > approach.
> >
> > Anyway, IIUC this v11 patch actually uses reg["config"] to compute the
> > offset, not 'ranges', so we should probably update the subject and
> > commit log to reflect that, and maybe remove the now-unused bits of
> > the devicetree example.
>
> We use reg["config"] to detect offset, but still need parent dts's ranges.
> There are two ranges, one is at parent pci bus [1], the other is under
> 'pci bus' [2].
Beside, luckly dwc use reg["config"] to indicate config space. but dt also
define ranges [2] under pcie node, which can also include 'config's space.
cadence also use reg["cfg"] to do that.
res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "cfg");
I am not sure why both choose use reg[] instead of [2]ranges under
pcie node. But the result make our situation simpler.
Frank
>
> Although use reg["config"], but still need ranges [1]. And information at
> ranges [2] also need be correct.
>
> The whole devicetree example also validate to help write address translate
> informaiton.
>
> >
> > I do worry a little bit about the assumption that the offset of
> > reg["config"] is the same as the offset of the other pieces. The main
> > place we use the offset on RCs is for the ATU, and isn't the ATU in
> > the MemSpace area at 0x8000_0000 below?
>
> No, "Bus fabric" only decode input address from "0x7000_0000..UPLIMIT".
> Then output address to 0x8000_0000..UPLIMIT. So below 0x8000_0000 never
> happen.
>
> >
> > It's great that in this case the 0x7ff0_0000 to 0x8ff0_0000 "config"
> > offset is the same as the 0x7000_0000 to 0x8000_0000 MemSpace offset,
> > but I don't know that this is guaranteed for all designs.
>
> So far, it is the same for use dwc chips. If we meet difference, we can
> add later.
>
> reg["config"] only simplied our implement base on the offset is the same.
> But whole concept is unchanged.
>
> Frank
>
> >
> > v9:
> > [PATCH v9 3/7] PCI: Add parent_bus_offset to resource_entry
> > https://lore.kernel.org/r/20250128-pci_fixup_addr-v9-3-3c4bb506f665@xxxxxxx
> > [PATCH v9 5/7] PCI: dwc: ep: Add parent_bus_addr for outbound window
> > https://lore.kernel.org/r/20250128-pci_fixup_addr-v9-5-3c4bb506f665@xxxxxxx
> >
> > v10:
> > [PATCH v10 05/10] PCI: dwc: Use devicetree 'ranges' property to get rid of cpu_addr_fixup() callback
> > https://lore.kernel.org/r/20250310-pci_fixup_addr-v10-5-409dafc950d1@xxxxxxx
> > [PATCH v10 06/10] PCI: dwc: ep: Add parent_bus_addr for outbound window
> > https://lore.kernel.org/r/20250310-pci_fixup_addr-v10-6-409dafc950d1@xxxxxxx
> >
> > > See below diagram:
> > >
> > > ┌─────────┐ ┌────────────┐
> > > ┌─────┐ │ │ IA: 0x8ff8_0000 │ │
> > > │ CPU ├───►│ ┌────►├─────────────────┐ │ PCI │
> > > └─────┘ │ │ │ IA: 0x8ff0_0000 │ │ │
> > > CPU Addr │ │ ┌─►├─────────────┐ │ │ Controller │
> > > 0x7ff8_0000─┼───┘ │ │ │ │ │ │
> > > │ │ │ │ │ │ │ PCI Addr
> > > 0x7ff0_0000─┼──────┘ │ │ └──► IOSpace ─┼────────────►
> > > │ │ │ │ │ 0
> > > 0x7000_0000─┼────────►├─────────┐ │ │ │
> > > └─────────┘ │ └──────► CfgSpace ─┼────────────►
> > > BUS Fabric │ │ │ 0
> > > │ │ │
> > > └──────────► MemSpace ─┼────────────►
> > > IA: 0x8000_0000 │ │ 0x8000_0000
> > > └────────────┘
> > >
> > > bus@5f000000 {
> > > compatible = "simple-bus";
> > > #address-cells = <1>;
> > > #size-cells = <1>;
> > > ranges = <0x80000000 0x0 0x70000000 0x10000000>;
> > >
> > > pcie@5f010000 {
> > > compatible = "fsl,imx8q-pcie";
> > > reg = <0x5f010000 0x10000>, <0x8ff00000 0x80000>;
> > > reg-names = "dbi", "config";
> > > #address-cells = <3>;
> > > #size-cells = <2>;
> > > device_type = "pci";
> > > bus-range = <0x00 0xff>;
> > > ranges = <0x81000000 0 0x00000000 0x8ff80000 0 0x00010000>,
> > > <0x82000000 0 0x80000000 0x80000000 0 0x0ff00000>;
> > > ...
> > > };
> > > };
> > >
> > > Term Intermediate address (IA) here means the address just before PCIe
> > > controller. After ATU use this IA instead CPU address, cpu_addr_fixup() can
> > > be removed.
> > >
> > > Use reg-name "config" to detect parent_bus_addr_offset. Suppose the offset
> > > is the same for all kinds of address translation.
> > >
> > > Just set parent_bus_offset, but doesn't use it, so no functional change
> > > intended yet.
> >
> > > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> > > @@ -474,6 +474,15 @@ int dw_pcie_host_init(struct dw_pcie_rp *pp)
> > > pp->io_base = pci_pio_to_address(win->res->start);
> > > }
> > >
> > > + /*
> > > + * visconti_pcie_cpu_addr_fixup() use pp->io_base,
> > > + * so have to call dw_pcie_init_parent_bus_offset() after init
> > > + * pp->io_base.
> > > + */
> > > + ret = dw_pcie_init_parent_bus_offset(pci, "config", pp->cfg0_base);
> > > + if (ret)
> > > + return ret;
> > > +
> > > /* Set default bus ops */
> > > bridge->ops = &dw_pcie_ops;
> > > bridge->child_ops = &dw_child_pcie_ops;
> > >
> > > --
> > > 2.34.1
> > >
