Hello! On Wednesday 18 January 2023 20:35:50 Tianfei Zhang wrote: > This patchset introduces the FPGA hotplug manager (fpgahp) driver which > has been verified on the Intel N3000 card. > > When a PCIe-based FPGA card is reprogrammed, it temporarily disappears > from the PCIe bus. This needs to be managed to avoid PCIe errors and to > reprobe the device after reprogramming. > > To change the FPGA image, the kernel burns a new image into the flash on > the card, and then triggers the card BMC to load the new image into FPGA. > A new FPGA hotplug manager driver is introduced that leverages the PCIe > hotplug framework to trigger and manage the update of the FPGA image, > including the disappearance and reappearance of the card on the PCIe bus. > The fpgahp driver uses APIs from the pciehp driver. Just I'm thinking about one thing. PCIe cards can support PCIe hotplug mechanism (via standard PCIe capabilities). So what would happen when FPGA based PCIe card is also hotplug-able? Will be there two PCI hotplug drivers/devices (one fpgahp and one pciehp)? Or just one and which? > Two new operation > callbacks are defined in hotplug_slot_ops: > > - available_images: Optional: available FPGA images > - image_load: Optional: trigger the FPGA to load a new image > > > The process of reprogramming an FPGA card begins by removing all devices > associated with the card that are not required for the reprogramming of > the card. This includes PCIe devices (PFs and VFs) associated with the > card as well as any other types of devices (platform, etc.) defined within > the FPGA. The remaining devices are referred to here as "reserved" devices. > After triggering the update of the FPGA card, the reserved devices are also > removed. > > The complete process for reprogramming the FPGA are: > 1. remove all PFs and VFs except for PF0 (reserved). > 2. remove all non-reserved devices of PF0. > 3. trigger FPGA card to do the image update. > 4. disable the link of the hotplug bridge. > 5. remove all reserved devices under hotplug bridge. > 6. wait for image reload done via BMC, e.g. 10s. > 7. re-enable the link of hotplug bridge > 8. enumerate PCI devices below the hotplug bridge > > usage example: > [root@localhost]# cd /sys/bus/pci/slot/X-X/ > > Get the available images. > [root@localhost 2-1]# cat available_images > bmc_factory bmc_user retimer_fw > > Load the request images for FPGA Card, for example load the BMC user image: > [root@localhost 2-1]# echo bmc_user > image_load > > Tianfei Zhang (12): > PCI: hotplug: add new callbacks on hotplug_slot_ops > PCI: hotplug: expose APIs from pciehp driver > PCI: hotplug: add and expose link disable API > PCI: hotplug: add FPGA PCI hotplug manager driver > fpga: dfl: register dfl-pci device into fpgahph driver > driver core: expose device_is_ancestor() API > PCI: hotplug: add register/unregister function for BMC device > fpga: m10bmc-sec: register BMC device into fpgahp driver > fpga: dfl: remove non-reserved devices > PCI: hotplug: implement the hotplug_slot_ops callback for fpgahp > fpga: m10bmc-sec: add m10bmc_sec_retimer_load callback > Documentation: fpga: add description of fpgahp driver > > Documentation/ABI/testing/sysfs-driver-fpgahp | 21 + > Documentation/fpga/fpgahp.rst | 29 + > Documentation/fpga/index.rst | 1 + > MAINTAINERS | 10 + > drivers/base/core.c | 3 +- > drivers/fpga/Kconfig | 2 + > drivers/fpga/dfl-pci.c | 95 +++- > drivers/fpga/dfl.c | 58 ++ > drivers/fpga/dfl.h | 4 + > drivers/fpga/intel-m10-bmc-sec-update.c | 246 ++++++++ > drivers/pci/hotplug/Kconfig | 14 + > drivers/pci/hotplug/Makefile | 1 + > drivers/pci/hotplug/fpgahp.c | 526 ++++++++++++++++++ > drivers/pci/hotplug/pci_hotplug_core.c | 88 +++ > drivers/pci/hotplug/pciehp.h | 3 + > drivers/pci/hotplug/pciehp_hpc.c | 11 +- > drivers/pci/hotplug/pciehp_pci.c | 2 + > include/linux/device.h | 1 + > include/linux/fpga/fpgahp_manager.h | 100 ++++ > include/linux/mfd/intel-m10-bmc.h | 31 ++ > include/linux/pci_hotplug.h | 5 + > 21 files changed, 1243 insertions(+), 8 deletions(-) > create mode 100644 Documentation/ABI/testing/sysfs-driver-fpgahp > create mode 100644 Documentation/fpga/fpgahp.rst > create mode 100644 drivers/pci/hotplug/fpgahp.c > create mode 100644 include/linux/fpga/fpgahp_manager.h > > > base-commit: 5dc4c995db9eb45f6373a956eb1f69460e69e6d4 > -- > 2.38.1 >
