On Mon, Mar 03, 2025 at 04:57:29PM -0500, Frank Li wrote:
> On Wed, Feb 26, 2025 at 06:23:26PM -0600, Bjorn Helgaas wrote:
> > On Tue, Jan 28, 2025 at 05:07:36PM -0500, Frank Li wrote:
> > > Introduce `parent_bus_offset` in `resource_entry` and a new API,
> > > `pci_add_resource_parent_bus_offset()`, to provide necessary information
> > > for PCI controllers with address translation units.
> > >
> > > Typical PCI data flow involves:
> > > CPU (CPU address) -> Bus Fabric (Intermediate address) ->
> > > PCI Controller (PCI bus address) -> PCI Bus.
> > >
> > > While most bus fabrics preserve address consistency, some modify addresses
> > > to intermediate values. The `parent_bus_offset` enables PCI controllers to
> > > translate these intermediate addresses correctly to PCI bus addresses.
> > >
> > > Pave the road to remove hardcoded cpu_addr_fixup() and similar patterns in
> > > PCI controller drivers.
> > > ...
> >
> > > +++ b/drivers/pci/of.c
> > > @@ -402,7 +402,17 @@ static int devm_of_pci_get_host_bridge_resources(struct device *dev,
> > > res->flags &= ~IORESOURCE_MEM_64;
> > > }
> > >
> > > - pci_add_resource_offset(resources, res, res->start - range.pci_addr);
> > > + /*
> > > + * IORESOURCE_IO res->start is io space start address.
> > > + * IORESOURCE_MEM res->start is cpu start address, which is the
> > > + * same as range.cpu_addr.
> > > + *
> > > + * Use (range.cpu_addr - range.parent_bus_addr) to align both
> > > + * IO and MEM's parent_bus_offset always offset to cpu address.
> > > + */
> > > +
> > > + pci_add_resource_parent_bus_offset(resources, res, res->start - range.pci_addr,
> > > + range.cpu_addr - range.parent_bus_addr);
> >
> > I don't know exactly where it needs to go, but I think we can call
> > .cpu_addr_fixup() once at startup on the base of the region. This
> > will tell us the offset that applies to the entire region, i.e.,
> > parent_bus_offset.
> >
> > Then we can remove all the .cpu_addr_fixup() calls in
> > cdns_pcie_host_init_address_translation(),
> > cdns_pcie_set_outbound_region(), and dw_pcie_prog_outbound_atu().
> >
> > Until we can get rid of all the .cpu_addr_fixup() implementations,
> > We'll still have that single call at startup (I guess once for cadence
> > and another for designware), but it should simplify the current
> > callers quite a bit.
>
> I don't think it can simple code. cdns_pcie_set_outbound_region() and
> dw_pcie_prog_outbound_atu() are called by EP functions, which have not use
> "resource" to manage outbound windows.
Let's ignore cadence for now. I don't think we need to solve that
until later.
dw_pcie_prog_outbound_atu() is called by:
- dw_pcie_other_conf_map_bus(): atu.parent_bus_addr = pp->cfg0_base
I think dw_pcie_host_init() can set pp->cfg0_base with the correct
intermediate address, either via the the of_property_read_reg() or
.cpu_addr_fixup().
If dw_pcie_host_init() does this, then we don't need
.cpu_addr_fixup() in dw_pcie_prog_outbound_atu().
- dw_pcie_rd_other_conf(): atu.parent_bus_addr = pp->io_base
Similarly, dw_pcie_host_init() should be able to set pp->io_base
to the intermediate address, so we don't need .cpu_addr_fixup() in
dw_pcie_prog_outbound_atu().
- dw_pcie_iatu_setup(): atu.parent_bus_addr = entry->res->start
Here "entry" iterates through bridge->windows, and we should be
able to set entry->parent_bus_offset at init-time, using
.cpu_addr_fixup() if necessary, so we can apply that offset
unconditionally, regardless of use_parent_dt_ranges, and we won't
need .cpu_addr_fixup() in dw_pcie_prog_outbound_atu().
- dw_pcie_pme_turn_off:
atu.parent_bus_addr = pci->pp.msg_res->start - pci->pp.msg_parent_bus_offset
This should be the same as dw_pcie_iatu_setup() since
msg_parent_bus_offset comes from the window iteration in
dw_pcie_host_request_msg_tlp_res(). As long as the windows have
the correct parent_bus_offset at init-time, we should be all set.
