A "VFIO variant" driver is a kernel driver for a device that supports all the APIs of the basic vfio-pci driver (which enables assigning a host PCI device to a QEMU guest) plus "some extra stuff" (e.g. to enable things like saving/restoring device state in order to support live migration.) Way back last year I posted a couple attempts to support VFIO variant drivers; here is V2 (along with a later followup discussion from a couple months ago): https://listman.redhat.com/archives/libvir-list/2022-August/233661.html https://listman.redhat.com/archives/libvir-list/2023-May/240108.html The mlx5-vfio-pci driver has now been upstream for quite awhile (and even in the downstream Fedora 38 kernel, for example), as are the sysfs bits that allow us to determine whether or not a driver is a VFIO variant, and I've updated the patch(es) to use this. I've also been working on auto-binding to the "best-match" VFIO variant driver based on comparing the device's modalias file in sysfs to the contents of the kernel's modules.alias file, but that isn't quite ready (partly code that isn't yet working, but also partly indecision about exactly where in the XML to put the driver name when it is specified; I won't take up more space here with that though). In the meantime, there are people who want to use the mlx5-vfio-pci driver (and Cedric Le Goater also has written vfio-pci-igbvf and vfio-pci-e1000e drivers (which area very useful for testing), although I don't think he has posted them anywhere yet), so I would like to get the basic patches here merged in upstream now while I continue working on "Part 2". These patches provide two improvements that make testing/using VFIO drivers much more convenient: 1) The specific driver can be given in the virsh nodedev-detach command (or the virNodeDeviceDetachFlags() API call), e.g.: virsh nodedev-detach pci_0000_04_11_5 --driver vfio-pci-igbvf 2) If the <hostdev> (or "<interface> ... <type='hostdev'/>" has "managed='no'", then libvirt will recognize any VFIO variant driver (rather than the current behavior of rejecting anything that isn't exactly "vfio-pci") With these two capabilities, it's simple and straightforward to bind a device to a VFIO variant driver, and then start a guest that uses that device. Change in V2: * complete remake, more refactoring * use existence of "vfio-dev" subdirectory of device directory in sysfs to determine whether the currently-bound driver is a vfio variant. * support binding to a user-specified driver during nodedev-detach, rather than only supporting vfio-pci. Laine Stump (8): util: use "stubDriverType" instead of just "stubDriver" util: add stub driver name to virPCIDevice object util: rename virPCIDeviceGetDriverPathAndName util: permit existing binding to VFIO variant driver util: probe stub driver from within function that binds to stub driver util: honor stubDriverName when probing/binding stub driver for a device node_device: support binding other drivers with virNodeDeviceDetachFlags() qemu: turn two multiline log messages into single line src/hypervisor/domain_driver.c | 9 +- src/hypervisor/domain_driver.h | 2 + src/hypervisor/virhostdev.c | 35 +++----- src/libvirt_private.syms | 9 +- src/libxl/libxl_driver.c | 3 +- src/qemu/qemu_driver.c | 37 ++++---- src/util/virnvme.c | 2 +- src/util/virpci.c | 156 +++++++++++++++++++++++++-------- src/util/virpci.h | 18 ++-- tests/virhostdevtest.c | 2 +- tests/virpcitest.c | 10 +-- 11 files changed, 185 insertions(+), 98 deletions(-) -- 2.41.0
