On Wed, Feb 28, 2024 at 12:07:06PM -0700, Alex Williamson wrote: > On Mon, 26 Feb 2024 15:24:58 -0700 > Alex Williamson <alex.williamson@xxxxxxxxxx> wrote: > > > On Mon, 26 Feb 2024 15:49:52 -0400 > > Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > > > > On Mon, Feb 26, 2024 at 12:41:07PM -0700, Alex Williamson wrote: > > > > On Mon, 26 Feb 2024 15:12:20 -0400 > > > > libvirt recently implemented support for managed="yes" with variant > > > > drivers where it will find the best "vfio_pci" driver for a device > > > > using an algorithm like Max suggested, but in practice it's not clear > > > > how useful that will be considering devices likes CX7 require > > > > configuration before binding to the variant driver. libvirt has no > > > > hooks to specify or perform configuration at that point. > > > > > > I don't think this is fully accurate (or at least not what was > > > intended), the VFIO device can be configured any time up until the VM > > > mlx5 driver reaches the device startup. > > > > > > Is something preventing this? Did we accidentally cache the migratable > > > flag in vfio or something?? > > > > I don't think so, I think this was just the policy we had decided > > relative to profiling VFs when they're created rather than providing a > > means to do it though a common vfio variant driver interface[1]. > > Turns out that yes, migration support needs to be established at probe > time. vfio_pci_core_register_device() expects migration_flags, > mig_ops, and log_ops to all be established by this point, which for > mlx5-vfio-pci occurs when the .init function calls > mlx5vf_cmd_set_migratable(). This is unfortunate, we should look at trying to accomodate this, IMHO. Yishai? > That also makes me wonder what happens if migration support is disabled > via devlink after binding the VF to mlx5-vfio-pci. Arguably this could > be considered user error, but what's the failure mode and support > implication? Thanks, I think tThe FW will start failing the migration commands. Jason
