On Fri, Mar 12, 2021 at 05:50:34AM +0900, Keith Busch wrote: > On Thu, Mar 11, 2021 at 04:22:34PM -0400, Jason Gunthorpe wrote: > > On Thu, Mar 11, 2021 at 12:16:02PM -0700, Keith Busch wrote: > > > On Thu, Mar 11, 2021 at 12:17:29PM -0600, Bjorn Helgaas wrote: > > > > On Wed, Mar 10, 2021 at 03:34:01PM -0800, Alexander Duyck wrote: > > > > > > > > > > I'm not so much worried about management software as the fact that > > > > > this is a vendor specific implementation detail that is shaping how > > > > > the kernel interfaces are meant to work. Other than the mlx5 I don't > > > > > know if there are any other vendors really onboard with this sort of > > > > > solution. > > > > > > > > I know this is currently vendor-specific, but I thought the value > > > > proposition of dynamic configuration of VFs for different clients > > > > sounded compelling enough that other vendors would do something > > > > similar. But I'm not an SR-IOV guy and have no vendor insight, so > > > > maybe that's not the case? > > > > > > NVMe has a similar feature defined by the standard where a PF controller can > > > dynamically assign MSIx vectors to VFs. The whole thing is managed in user > > > space with an ioctl, though. I guess we could wire up the driver to handle it > > > through this sysfs interface too, but I think the protocol specific tooling is > > > more appropriate for nvme. > > > > Really? Why not share a common uAPI? > > We associate interrupt vectors with other dynamically assigned nvme > specific resources (IO queues), and these are not always allocated 1:1. mlx5 is doing that too, the end driver gets to assign the MSI vector to a CPU and then dynamically attach queues to it. I'm not sure I get why nvme would want to link those two things as the CPU assignment and queue attach could happen in a VM while the MSIX should be in the host? > A common uAPI for MSIx only gets us half way to configuring the VFs for > that particular driver. > > > Do you have a standards reference for this? > > Yes, sections 5.22 and 8.5 from this spec: > > https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4a-2020.03.09-Ratified.pdf > > An example of open source tooling implementing this is nvme-cli's > "nvme virt-mgmt" command. Oh it is fascinating! 8.5.2 looks like exactly the same thing being implemented here for mlx5, including changing the "Read only" config space value Still confused why this shouldn't be the same API?? Jason
