Hi Alex, Cornelia, Jason and Co, This series split the vfio_pci driver into 2 parts: pci drivers and a subsystem driver that will also be library of code. The main pci driver, vfio_pci.ko will be used as before and it will bind to the subsystem driver vfio_pci_core.ko to register to the VFIO subsystem. This series is coming to solve some of the issues that were raised in the previous attempts for extending vfio-pci for vendor specific functionality: 1. https://lkml.org/lkml/2020/5/17/376 by Yan Zhao. 2. https://www.spinics.net/lists/kernel/msg3903996.html by Longfang Liu This subsystem framework will also ease on adding new vendor specific functionality to VFIO devices in the future by allowing another module to provide the pci_driver that can setup number of details before registering to VFIO subsystem (such as inject its own operations). This series also extends the "driver_override" mechanism. We added a flag for pci drivers that will declare themselves as "driver_override" capable and only declared drivers can use this mechanism in the PCI subsystem. Other drivers will not be able to bind to devices that use "driver_override". Also, the PCI driver matching will always look for ID table and will never generate dummy "match_all" ID table in the PCI subsystem layer. In this way, we ensure deterministic behaviour with no races with the original pci drivers. In order to get the best match for "driver_override" drivers, one can create a userspace program (example can be found at https://github.com/maxgurtovoy/linux_tools/blob/main/vfio/bind_vfio_pci_driver.py) that find the 'best match' according to simple algorithm: "the driver with the fewest '*' matches wins." For example, the vfio-pci driver will match to any pci device. So it will have the maximal '*' matches (for all matching IDs: vendor, device, subvendor, ...). In case we are looking for a match to mlx5 based device, we'll have a match to vfio-pci.ko and mlx5-vfio-pci.ko. We'll prefer mlx5-vfio-pci.ko since it will have less '*' matches (probably vendor and device IDs will match). This will work in the future for NVMe/Virtio devices that can match according to a class code or other criteria. The main goal of this series is to agree on the vfio_pci module split and the "driver_override" extensions. The follow-up version will include an extended mlx5_vfio_pci driver that will support VF suspend/resume as well. This series applied cleanly on top of vfio reflck re-design (still haven't sent for review) and can be found at: https://github.com/Mellanox/NVMEoF-P2P/tree/vfio-v4-external. Max Gurtovoy (11): vfio-pci: rename vfio_pci.c to vfio_pci_core.c vfio-pci: rename vfio_pci_private.h to vfio_pci_core.h vfio-pci: rename vfio_pci_device to vfio_pci_core_device vfio-pci: rename ops functions to fit core namings vfio-pci: include vfio header in vfio_pci_core.h vfio-pci: introduce vfio_pci.c vfio-pci: move igd initialization to vfio_pci.c PCI: add flags field to pci_device_id structure PCI: add matching checks for driver_override binding vfio-pci: introduce vfio_pci_core subsystem driver mlx5-vfio-pci: add new vfio_pci driver for mlx5 devices Documentation/ABI/testing/sysfs-bus-pci | 6 +- Documentation/PCI/pci.rst | 1 + drivers/pci/pci-driver.c | 22 +- drivers/vfio/pci/Kconfig | 27 +- drivers/vfio/pci/Makefile | 12 +- drivers/vfio/pci/mlx5_vfio_pci.c | 130 + drivers/vfio/pci/vfio_pci.c | 2329 +---------------- drivers/vfio/pci/vfio_pci_config.c | 70 +- drivers/vfio/pci/vfio_pci_core.c | 2239 ++++++++++++++++ drivers/vfio/pci/vfio_pci_igd.c | 16 +- drivers/vfio/pci/vfio_pci_intrs.c | 42 +- drivers/vfio/pci/vfio_pci_rdwr.c | 18 +- drivers/vfio/pci/vfio_pci_zdev.c | 4 +- include/linux/mod_devicetable.h | 9 + include/linux/pci.h | 27 + .../linux/vfio_pci_core.h | 93 +- scripts/mod/devicetable-offsets.c | 1 + scripts/mod/file2alias.c | 8 +- 18 files changed, 2695 insertions(+), 2359 deletions(-) create mode 100644 drivers/vfio/pci/mlx5_vfio_pci.c create mode 100644 drivers/vfio/pci/vfio_pci_core.c rename drivers/vfio/pci/vfio_pci_private.h => include/linux/vfio_pci_core.h (56%) -- 2.21.0
