This patchset introduces the FPGA hotplug manager (fpgahp) driver which
has been verified on the Intel N3000 card.
When a PCIe-based FPGA card is reprogrammed, it temporarily disappears
from the PCIe bus. This needs to be managed to avoid PCIe errors and to
reprobe the device after reprogramming.
To change the FPGA image, the kernel burns a new image into the flash on
the card, and then triggers the card BMC to load the new image into FPGA.
A new FPGA hotplug manager driver is introduced that leverages the PCIe
hotplug framework to trigger and manage the update of the FPGA image,
including the disappearance and reappearance of the card on the PCIe bus.
The fpgahp driver uses APIs from the pciehp driver. Two new operation
callbacks are defined in hotplug_slot_ops:
- available_images: Optional: available FPGA images
- image_load: Optional: trigger the FPGA to load a new image
The process of reprogramming an FPGA card begins by removing all devices
associated with the card that are not required for the reprogramming of
the card. This includes PCIe devices (PFs and VFs) associated with the
card as well as any other types of devices (platform, etc.) defined within
the FPGA. The remaining devices are referred to here as "reserved" devices.
After triggering the update of the FPGA card, the reserved devices are also
removed.
The complete process for reprogramming the FPGA are:
1. remove all PFs and VFs except for PF0 (reserved).
2. remove all non-reserved devices of PF0.
3. trigger FPGA card to do the image update.
4. disable the link of the hotplug bridge.
5. remove all reserved devices under hotplug bridge.
6. wait for image reload done via BMC, e.g. 10s.
7. re-enable the link of hotplug bridge
8. enumerate PCI devices below the hotplug bridge
usage example:
[root@localhost]# cd /sys/bus/pci/slot/X-X/
Get the available images.
[root@localhost 2-1]# cat available_images
bmc_factory bmc_user retimer_fw
Load the request images for FPGA Card, for example load the BMC user image:
[root@localhost 2-1]# echo bmc_user > image_load
Tianfei Zhang (12):
PCI: hotplug: add new callbacks on hotplug_slot_ops
PCI: hotplug: expose APIs from pciehp driver
PCI: hotplug: add and expose link disable API
PCI: hotplug: add FPGA PCI hotplug manager driver
fpga: dfl: register dfl-pci device into fpgahph driver
driver core: expose device_is_ancestor() API
PCI: hotplug: add register/unregister function for BMC device
fpga: m10bmc-sec: register BMC device into fpgahp driver
fpga: dfl: remove non-reserved devices
PCI: hotplug: implement the hotplug_slot_ops callback for fpgahp
fpga: m10bmc-sec: add m10bmc_sec_retimer_load callback
Documentation: fpga: add description of fpgahp driver
Documentation/ABI/testing/sysfs-driver-fpgahp | 21 +
Documentation/fpga/fpgahp.rst | 29 +
Documentation/fpga/index.rst | 1 +
MAINTAINERS | 10 +
drivers/base/core.c | 3 +-
drivers/fpga/Kconfig | 2 +
drivers/fpga/dfl-pci.c | 95 +++-
drivers/fpga/dfl.c | 58 ++
drivers/fpga/dfl.h | 4 +
drivers/fpga/intel-m10-bmc-sec-update.c | 246 ++++++++
drivers/pci/hotplug/Kconfig | 14 +
drivers/pci/hotplug/Makefile | 1 +
drivers/pci/hotplug/fpgahp.c | 526 ++++++++++++++++++
drivers/pci/hotplug/pci_hotplug_core.c | 88 +++
drivers/pci/hotplug/pciehp.h | 3 +
drivers/pci/hotplug/pciehp_hpc.c | 11 +-
drivers/pci/hotplug/pciehp_pci.c | 2 +
include/linux/device.h | 1 +
include/linux/fpga/fpgahp_manager.h | 100 ++++
include/linux/mfd/intel-m10-bmc.h | 31 ++
include/linux/pci_hotplug.h | 5 +
21 files changed, 1243 insertions(+), 8 deletions(-)
create mode 100644 Documentation/ABI/testing/sysfs-driver-fpgahp
create mode 100644 Documentation/fpga/fpgahp.rst
create mode 100644 drivers/pci/hotplug/fpgahp.c
create mode 100644 include/linux/fpga/fpgahp_manager.h
base-commit: 5dc4c995db9eb45f6373a956eb1f69460e69e6d4
--
2.38.1
