On 3/21/23 03:44, Christian Gmeiner wrote: > Hi all > > Am Do., 9. März 2023 um 09:52 Uhr schrieb Clément Léger > <clement.leger@xxxxxxxxxxx>: >> >> Le Wed, 8 Mar 2023 01:31:52 -0600, >> Frank Rowand <frowand.list@xxxxxxxxx> a écrit : >> >>> On 3/6/23 18:52, Rob Herring wrote: >>>> On Mon, Mar 6, 2023 at 3:24 PM Frank Rowand <frowand.list@xxxxxxxxx> wrote: >>>>> >>> >>> < snip > >>> >>> Hi Rob, >>> >>> I am in no position to comment intelligently on your comments until I >>> understand the SoC on PCI card model I am asking to be described in >>> this subthread. >> >> Hi Frank, >> >> Rather than answering all of the assumptions that were made in the upper >> thread (that are probably doing a bit too much of inference), I will >> re-explain that from scratch. >> >> Our usecase involves the lan966x SoCs. These SoCs are mainly targeting >> networking application and offers multiple SFP and RGMII interfaces. >> This Soc can be used in two exclusive modes (at least for the intended >> usage): >> >> SoC mode: >> The device runs Linux by itself, on ARM64 cores included in the >> SoC. This use-case of the lan966x is currently almost upstreamed, >> using a traditional Device Tree representation of the lan996x HW >> blocks [1] A number of drivers for the different IPs of the SoC have >> already been merged in upstream Linux (see >> arch/arm/boot/dts/lan966x.dtsi) >> >> PCI mode: >> The lan966x SoC is configured as a PCIe endpoint (PCI card), >> connected to a separate platform that acts as the PCIe root complex. >> In this case, all the IO memories that are exposed by the devices >> embedded on this SoC are exposed through PCI BARs 0 & 1 and the ARM64 >> cores of the SoC are not used. Since this is a PCIe card, it can be >> plugged on any platform, of any architecture supporting PCIe. >> >> This work only focus on the *PCI mode* usage. In this mode, we have the >> following prerequisites: >> - Should work on all architectures (x86, ARM64, etc) >> - Should be self-contained in the driver >> - Should be able to reuse all existing platform drivers >> >> In PCI mode, the card runs a firmware (not that it matters at all by >> the way) which configure the card in PCI mode at boot time. In this >> mode, it exposes a single PCI physical function associated with >> vendor/product 0x1055/0x9660. This is not a multi-function PCI device ! >> This means that all the IO memories (peripheral memories, device >> memories, registers, whatever you call them) are accessible using >> standard readl()/writel() on the BARs that have been remapped. For >> instance (not accurate), in the BAR 0, we will have this kind of memory >> map: >> >> BAR0 >> 0x0 ┌───────────┐ >> │ │ >> ├───────────┤ >> │ Clock │ >> │ controller│ >> ├───────────┤ >> │ │ >> ├───────────┤ >> │ I2C │ >> │ controller│ >> ├───────────┤ >> │ │ >> ├───────────┤ >> │ MDIO │ >> │ Controller│ >> ├───────────┤ >> │ │ >> ├───────────┤ >> │ Switch │ >> │ Controller│ >> ├───────────┤ >> │ │ >> │ ... │ >> >> >> It also exposes either a single interrupt via the legacy interrupt >> (which can then be demuxed by reading the SoC internal interrupt >> controller registers), or multiple interrupts using MSI interrupts. >> >> As stated before, all these peripherals are already supported in SoC >> mode and thus, there are aleready existing platform drivers for each of >> them. For more information about the devices that are exposed please >> see link [1] which is the device-tree overlay used to describe the >> lan9662 card. >> >> In order to use the ethernet switch, we must configure everything that >> lies around this ethernet controller, here are a few amongst all of >> them: >> - MDIO bus >> - I2C controller for SFP modules access >> - Clock controller >> - Ethernet controller >> - Syscon >> >> Since all the platform drivers already exist for these devices, we >> want to reuse them. Multiple solutions were thought of (fwnode, mfd, >> ACPI, device-tree) and eventually ruled out for some of them and efforts >> were made to try to tackle that (using fwnode [2], device-tree [3]) >> >> One way to do so is to use a device-tree overlay description that is >> loaded dynamically on the PCI device OF node. This can be done using the >> various device-tree series series that have been proposed (included >> this one). On systems that do not provide a device-tree of_root, create >> an empty of_root node (see [4]). Then during PCI enumeration, create >> device-tree node matching the PCI tree that was enumerated (See [5]). >> This is needed since the PCI card can be plugged on whatever port the >> user wants and thus it can not be statically described using a fixed >> "target-path" property in the overlay. >> >> Finally, to glue everything together, we add a PCI driver for the >> VID/PID of the PCI card (See [6]). This driver is responsible of adding >> the "ranges" property in the device-tree PCI node to remap the child >> nodes "reg" property to the PCI memory map. This is needed because the >> PCI memory addresses differ between platform, enumeration order and so >> on.Finally, the driver will load the device-tree overlay (See [1]) to >> the PCI device-tree node. Eventually, a call to >> of_platform_default_populate() will probe the nodes and platform >> drivers. >> >> I hope this will help you understanding what is going on here. In the >> meantime, I'm also trying to obtain public documentation about the >> lan966x SoC. >> >> [1] >> https://github.com/clementleger/linux/blob/bf9b4ef803d86c4ae59a4ca195a4152b0d5c3cea/drivers/mfd/lan966x_pci.dts >> [2] >> https://lore.kernel.org/netdev/YhPSkz8+BIcdb72R@xxxxxxxxxxxxxxxxxx/T/ >> [3] >> https://lore.kernel.org/lkml/20220427094502.456111-1-clement.leger@xxxxxxxxxxx/ >> [4] >> https://lore.kernel.org/lkml/20230223213418.891942-1-frowand.list@xxxxxxxxx/ >> [5] >> https://lore.kernel.org/lkml/1674183732-5157-1-git-send-email-lizhi.hou@xxxxxxx/ >> [6] >> https://github.com/clementleger/linux/blob/bf9b4ef803d86c4ae59a4ca195a4152b0d5c3cea/drivers/mfd/lan966x_pci_of.c >> >> -- >> Clément Léger, >> Embedded Linux and Kernel engineer at Bootlin >> https://bootlin.com > > What is missing to move on with this patch set? I need to evaluate what Clément Léger wrote in the email you replied to. I had overlooked this reply to my questions. From a quick scan, it looks like he _probably_ provided the context I was looking for to understand the architecture of the proposal. But it is late Sunday night, so I won't get to this tonight. -Frank >
