On Wed, Feb 26, 2025 at 06:23:26PM -0600, Bjorn Helgaas wrote:
> On Tue, Jan 28, 2025 at 05:07:36PM -0500, Frank Li wrote:
> > Introduce `parent_bus_offset` in `resource_entry` and a new API,
> > `pci_add_resource_parent_bus_offset()`, to provide necessary information
> > for PCI controllers with address translation units.
> >
> > Typical PCI data flow involves:
> > CPU (CPU address) -> Bus Fabric (Intermediate address) ->
> > PCI Controller (PCI bus address) -> PCI Bus.
> >
> > While most bus fabrics preserve address consistency, some modify addresses
> > to intermediate values. The `parent_bus_offset` enables PCI controllers to
> > translate these intermediate addresses correctly to PCI bus addresses.
> >
> > Pave the road to remove hardcoded cpu_addr_fixup() and similar patterns in
> > PCI controller drivers.
> > ...
>
> > +++ b/drivers/pci/of.c
> > @@ -402,7 +402,17 @@ static int devm_of_pci_get_host_bridge_resources(struct device *dev,
> > res->flags &= ~IORESOURCE_MEM_64;
> > }
> >
> > - pci_add_resource_offset(resources, res, res->start - range.pci_addr);
> > + /*
> > + * IORESOURCE_IO res->start is io space start address.
> > + * IORESOURCE_MEM res->start is cpu start address, which is the
> > + * same as range.cpu_addr.
> > + *
> > + * Use (range.cpu_addr - range.parent_bus_addr) to align both
> > + * IO and MEM's parent_bus_offset always offset to cpu address.
> > + */
> > +
> > + pci_add_resource_parent_bus_offset(resources, res, res->start - range.pci_addr,
> > + range.cpu_addr - range.parent_bus_addr);
>
> I don't know exactly where it needs to go, but I think we can call
> .cpu_addr_fixup() once at startup on the base of the region. This
> will tell us the offset that applies to the entire region, i.e.,
> parent_bus_offset.
>
> Then we can remove all the .cpu_addr_fixup() calls in
> cdns_pcie_host_init_address_translation(),
> cdns_pcie_set_outbound_region(), and dw_pcie_prog_outbound_atu().
>
> Until we can get rid of all the .cpu_addr_fixup() implementations,
> We'll still have that single call at startup (I guess once for cadence
> and another for designware), but it should simplify the current
> callers quite a bit.
I don't think it can simple code. cdns_pcie_set_outbound_region() and
dw_pcie_prog_outbound_atu() are called by EP functions, which have not use
"resource" to manage outbound windows. And there are IO/CFG/MEM space at
host side, It will involve more .cpu_addr_fixup() calls to setup these
range.
Frank
>
> > }
> >
> > /* Check for dma-ranges property */
> > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > index 47b31ad724fa5..0d7e67b47be47 100644
> > --- a/include/linux/pci.h
> > +++ b/include/linux/pci.h
> > @@ -1510,6 +1510,8 @@ static inline void pci_release_config_region(struct pci_dev *pdev,
> > void pci_add_resource(struct list_head *resources, struct resource *res);
> > void pci_add_resource_offset(struct list_head *resources, struct resource *res,
> > resource_size_t offset);
> > +void pci_add_resource_parent_bus_offset(struct list_head *resources, struct resource *res,
> > + resource_size_t offset, resource_size_t parent_bus_offset);
> > void pci_free_resource_list(struct list_head *resources);
> > void pci_bus_add_resource(struct pci_bus *bus, struct resource *res);
> > struct resource *pci_bus_resource_n(const struct pci_bus *bus, int n);
> > diff --git a/include/linux/resource_ext.h b/include/linux/resource_ext.h
> > index ff0339df56afc..b6ec6cc318203 100644
> > --- a/include/linux/resource_ext.h
> > +++ b/include/linux/resource_ext.h
> > @@ -24,6 +24,7 @@ struct resource_entry {
> > struct list_head node;
> > struct resource *res; /* In master (CPU) address space */
> > resource_size_t offset; /* Translation offset for bridge */
> > + resource_size_t parent_bus_offset; /* Parent bus address offset for bridge */
> > struct resource __res; /* Default storage for res */
> > };
> >
> >
> > --
> > 2.34.1
> >
