On Mon, 26 Feb 2024 15:24:58 -0700 Alex Williamson <alex.williamson@xxxxxxxxxx> wrote: > On Mon, 26 Feb 2024 15:49:52 -0400 > Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > > On Mon, Feb 26, 2024 at 12:41:07PM -0700, Alex Williamson wrote: > > > On Mon, 26 Feb 2024 15:12:20 -0400 > > > libvirt recently implemented support for managed="yes" with variant > > > drivers where it will find the best "vfio_pci" driver for a device > > > using an algorithm like Max suggested, but in practice it's not clear > > > how useful that will be considering devices likes CX7 require > > > configuration before binding to the variant driver. libvirt has no > > > hooks to specify or perform configuration at that point. > > > > I don't think this is fully accurate (or at least not what was > > intended), the VFIO device can be configured any time up until the VM > > mlx5 driver reaches the device startup. > > > > Is something preventing this? Did we accidentally cache the migratable > > flag in vfio or something?? > > I don't think so, I think this was just the policy we had decided > relative to profiling VFs when they're created rather than providing a > means to do it though a common vfio variant driver interface[1]. Turns out that yes, migration support needs to be established at probe time. vfio_pci_core_register_device() expects migration_flags, mig_ops, and log_ops to all be established by this point, which for mlx5-vfio-pci occurs when the .init function calls mlx5vf_cmd_set_migratable(). So the VF does indeed need to be "profiled" to enabled migration prior to binding to the mlx5-vfio-pci driver in order to report support. That also makes me wonder what happens if migration support is disabled via devlink after binding the VF to mlx5-vfio-pci. Arguably this could be considered user error, but what's the failure mode and support implication? Thanks, Alex
