> From: Jason Gunthorpe <jgg@xxxxxxxx> > Sent: Thursday, December 8, 2022 4:08 AM > > For instance how do I trap FLR like mlx5 must do if the > drivers/live_migration code cannot intercept the FLR VFIO ioctl? > > How do I mangle and share the BAR like hisilicon does? > > Which is really why this is in VFIO in the first place. It actually is > coupled in practice, if not in theory. > Those are good questions which I also buy in to stay with the current VFIO framework. Actually the same controlling vs. controlled design choice also exists in the hardware side. There are plenty of SR-IOV devices supporting doorbells for VF (controlled function) to call services on PF (controlling function) while the doorbell interface is implemented on the VF side. If following the direction having controlling function to explicitly provide services then all those doorbells should be deprecated and instead we want explicit communications between VF driver and PF driver. >From userspace driver p.o.v. the VFIO uAPI is kind of a device programming interface. Here we just present everything related to the controlled device itself (including the state management) via a centralized portal though in the kernel there might be linkage out of VFIO to reach the controlling driver. kind of a sw doorbell. 😊 btw just to add more background to this work. Half a year ago Lei actually did a flavor [1] in the other way. The controlling function of Intel IPU card also supports a network gRPC protocol to manage the state of controlled NVMe function. Then that series attempted to introduce an out-of-band migration framework in Qemu so instead of doing in-line state management via kernel VFIO uAPI Qemu can turn to an external plugin which forwards the state cmd via gRPC to the controlling function. Just that the controlling driver is not in the kernel. It's dropped as the inline way was preferred. Thanks Kevin [1] https://lore.kernel.org/all/20220524061848.1615706-14-lei.rao@xxxxxxxxx/T/
