The Linux kernel has recently added support for device-specific VFIO
drivers to be used (instead of the current generic, one-size-fits-all
vfio_pci driver) when assigning a device to a guest with VFIO. The
intent of this is to (for example) support APIs for device-specific
setup that needs to be done before the device is handed over to the
guest, and to support migration of guests using these devices. More
details can be found in the comments of the patch emails here:
https://lore.kernel.org/kvm/20210826103912.128972-1-yishaih@xxxxxxxxxx/
There are a couple of issues that need to be resolved before that will
work well with libvirt-managed guests:
1) Although they've outlined a system for determining the best / most
specific driver for a device by extending the existing modules aliases,
the kernel people have left it up to userspace to actually parse the
module.aliases info to determine the best driver for a device. They've
even "thrown us a bone" in the form of an example python script to do
just that:
https://github.com/maxgurtovoy/linux_tools/blob/main/vfio/bind_vfio_pci_driver.py
Currently, if a device is assigned to a guest with <hostdev
managed='yes'> (the default) then libvirt will unbind the device from
whatever host driver its bound to, and bind it to the vfio_pci driver
(since, up to now, that has been the only driver available). In the
future, if we want to be able to take advantage of the extended
capabilities provided by the device-specific vfio drivers, we will need
to figure out which driver to load.
I personally don't like the idea of embedding the logic from the above
example program into libvirt - this really seems to me like something
that should be implemented in one place for use by many consumers
(libvirt being one of the consumers). One suggestion made by Alex
Williamson (in a private discussion, not on a mailing list) was that
possibly we could get the driverctl utility to add this functionality,
and then call driverctl to learn the name of the best-fit vfio driver
for a device. I haven't yet looked at it, but I've been told that
driverctl is a bash script :-O. If we decide that's a good route,
perhaps we could also convince someone to convert drivertctl to rust,
similar to what jjongsma did with mdevctl (nudge nudge, hint hint).
Or maybe I'm making it into a bigger deal than it needs to be, and we
should just implement the logic of the python script up above directly
in libvirt. Does anyone have an opinion?
(NB: If a device is assigned to a guest by libvirt with <hostdev
managed='no'> then libvirt will just blindly assume that it is already
bound to the correct driver, so until we get internal support for using
the device-specific drivers, users can still get the same functionality
by binding their devices to the device-specific vfio driver externally
to libvirt).
=====
2) There may be cases where, in spite of a device-specific driver being
available, the user prefers to use the generic vfio_pci driver. To
support that we will need to have a place in our config to set the
driver name.
We already have a <driver> subelement of <hostdev> that was originally
added to allow choosing between VFIO device assignment and legacy KVM
device assignment:
<driver name='vfio|kvm'/>
All support for KVM device assignment was removed a few years ago, so in
practice the driver name is always "vfio". The most natural looking way
to support device-specific drivers would be to use this name attribute
to specify the driver name, e.g. if you wanted to let libvirt select the
best driver, you would use:
<driver name='vfio'/>
(what it's currently set to in everyone's configs). But if you wanted to
force use of the generic driver, you'd use:
<driver name='vfio_pci'/>
or if you wanted to force use of another driver that wasn't the 'best
fit' according to the module aliases, you could use, e.g.:
<driver name='vfio_pci_xyzzy'/>
I'm uncomfortable with the fact that we're effectively "re-using" the
name attribute for a new purpose though - up until now it has been
"which device assignment method?", but this changes it to "which vfio
driver should be loaded?". So maybe we need a new element. I'm not sure
what to call it though. How about this?
<driver name='vfio' modname='vfio_pci'/>
Does anyone have a better idea for the name?
