Hi all Am Do., 9. März 2023 um 09:52 Uhr schrieb Clément Léger <clement.leger@xxxxxxxxxxx>: > > Le Wed, 8 Mar 2023 01:31:52 -0600, > Frank Rowand <frowand.list@xxxxxxxxx> a écrit : > > > On 3/6/23 18:52, Rob Herring wrote: > > > On Mon, Mar 6, 2023 at 3:24 PM Frank Rowand <frowand.list@xxxxxxxxx> wrote: > > >> > > > > < snip > > > > > Hi Rob, > > > > I am in no position to comment intelligently on your comments until I > > understand the SoC on PCI card model I am asking to be described in > > this subthread. > > Hi Frank, > > Rather than answering all of the assumptions that were made in the upper > thread (that are probably doing a bit too much of inference), I will > re-explain that from scratch. > > Our usecase involves the lan966x SoCs. These SoCs are mainly targeting > networking application and offers multiple SFP and RGMII interfaces. > This Soc can be used in two exclusive modes (at least for the intended > usage): > > SoC mode: > The device runs Linux by itself, on ARM64 cores included in the > SoC. This use-case of the lan966x is currently almost upstreamed, > using a traditional Device Tree representation of the lan996x HW > blocks [1] A number of drivers for the different IPs of the SoC have > already been merged in upstream Linux (see > arch/arm/boot/dts/lan966x.dtsi) > > PCI mode: > The lan966x SoC is configured as a PCIe endpoint (PCI card), > connected to a separate platform that acts as the PCIe root complex. > In this case, all the IO memories that are exposed by the devices > embedded on this SoC are exposed through PCI BARs 0 & 1 and the ARM64 > cores of the SoC are not used. Since this is a PCIe card, it can be > plugged on any platform, of any architecture supporting PCIe. > > This work only focus on the *PCI mode* usage. In this mode, we have the > following prerequisites: > - Should work on all architectures (x86, ARM64, etc) > - Should be self-contained in the driver > - Should be able to reuse all existing platform drivers > > In PCI mode, the card runs a firmware (not that it matters at all by > the way) which configure the card in PCI mode at boot time. In this > mode, it exposes a single PCI physical function associated with > vendor/product 0x1055/0x9660. This is not a multi-function PCI device ! > This means that all the IO memories (peripheral memories, device > memories, registers, whatever you call them) are accessible using > standard readl()/writel() on the BARs that have been remapped. For > instance (not accurate), in the BAR 0, we will have this kind of memory > map: > > BAR0 > 0x0 ┌───────────┐ > │ │ > ├───────────┤ > │ Clock │ > │ controller│ > ├───────────┤ > │ │ > ├───────────┤ > │ I2C │ > │ controller│ > ├───────────┤ > │ │ > ├───────────┤ > │ MDIO │ > │ Controller│ > ├───────────┤ > │ │ > ├───────────┤ > │ Switch │ > │ Controller│ > ├───────────┤ > │ │ > │ ... │ > > > It also exposes either a single interrupt via the legacy interrupt > (which can then be demuxed by reading the SoC internal interrupt > controller registers), or multiple interrupts using MSI interrupts. > > As stated before, all these peripherals are already supported in SoC > mode and thus, there are aleready existing platform drivers for each of > them. For more information about the devices that are exposed please > see link [1] which is the device-tree overlay used to describe the > lan9662 card. > > In order to use the ethernet switch, we must configure everything that > lies around this ethernet controller, here are a few amongst all of > them: > - MDIO bus > - I2C controller for SFP modules access > - Clock controller > - Ethernet controller > - Syscon > > Since all the platform drivers already exist for these devices, we > want to reuse them. Multiple solutions were thought of (fwnode, mfd, > ACPI, device-tree) and eventually ruled out for some of them and efforts > were made to try to tackle that (using fwnode [2], device-tree [3]) > > One way to do so is to use a device-tree overlay description that is > loaded dynamically on the PCI device OF node. This can be done using the > various device-tree series series that have been proposed (included > this one). On systems that do not provide a device-tree of_root, create > an empty of_root node (see [4]). Then during PCI enumeration, create > device-tree node matching the PCI tree that was enumerated (See [5]). > This is needed since the PCI card can be plugged on whatever port the > user wants and thus it can not be statically described using a fixed > "target-path" property in the overlay. > > Finally, to glue everything together, we add a PCI driver for the > VID/PID of the PCI card (See [6]). This driver is responsible of adding > the "ranges" property in the device-tree PCI node to remap the child > nodes "reg" property to the PCI memory map. This is needed because the > PCI memory addresses differ between platform, enumeration order and so > on.Finally, the driver will load the device-tree overlay (See [1]) to > the PCI device-tree node. Eventually, a call to > of_platform_default_populate() will probe the nodes and platform > drivers. > > I hope this will help you understanding what is going on here. In the > meantime, I'm also trying to obtain public documentation about the > lan966x SoC. > > [1] > https://github.com/clementleger/linux/blob/bf9b4ef803d86c4ae59a4ca195a4152b0d5c3cea/drivers/mfd/lan966x_pci.dts > [2] > https://lore.kernel.org/netdev/YhPSkz8+BIcdb72R@xxxxxxxxxxxxxxxxxx/T/ > [3] > https://lore.kernel.org/lkml/20220427094502.456111-1-clement.leger@xxxxxxxxxxx/ > [4] > https://lore.kernel.org/lkml/20230223213418.891942-1-frowand.list@xxxxxxxxx/ > [5] > https://lore.kernel.org/lkml/1674183732-5157-1-git-send-email-lizhi.hou@xxxxxxx/ > [6] > https://github.com/clementleger/linux/blob/bf9b4ef803d86c4ae59a4ca195a4152b0d5c3cea/drivers/mfd/lan966x_pci_of.c > > -- > Clément Léger, > Embedded Linux and Kernel engineer at Bootlin > https://bootlin.com What is missing to move on with this patch set? -- greets -- Christian Gmeiner, MSc https://christian-gmeiner.info/privacypolicy
