On Mon, Mar 01, 2021 at 09:55:21AM +0200, Leon Romanovsky wrote: > From: Leon Romanovsky <leonro@xxxxxxxxxx> > > A typical cloud provider SR-IOV use case is to create many VFs for use by > guest VMs. The VFs may not be assigned to a VM until a customer requests a > VM of a certain size, e.g., number of CPUs. A VF may need MSI-X vectors > proportional to the number of CPUs in the VM, but there is no standard way > to change the number of MSI-X vectors supported by a VF. > > Some Mellanox ConnectX devices support dynamic assignment of MSI-X vectors > to SR-IOV VFs. This can be done by the PF driver after VFs are enabled, > and it can be done without affecting VFs that are already in use. The > hardware supports a limited pool of MSI-X vectors that can be assigned to > the PF or to individual VFs. This is device-specific behavior that > requires support in the PF driver. > > Add a read-only "sriov_vf_total_msix" sysfs file for the PF and a writable > "sriov_vf_msix_count" file for each VF. Management software may use these > to learn how many MSI-X vectors are available and to dynamically assign > them to VFs before the VFs are passed through to a VM. > > If the PF driver implements the ->sriov_get_vf_total_msix() callback, > "sriov_vf_total_msix" contains the total number of MSI-X vectors available > for distribution among VFs. > > If no driver is bound to the VF, writing "N" to "sriov_vf_msix_count" uses > the PF driver ->sriov_set_msix_vec_count() callback to assign "N" MSI-X > vectors to the VF. When a VF driver subsequently reads the MSI-X Message > Control register, it will see the new Table Size "N". > > Signed-off-by: Leon Romanovsky <leonro@xxxxxxxxxx> > --- > Documentation/ABI/testing/sysfs-bus-pci | 29 +++++++ > drivers/pci/iov.c | 102 ++++++++++++++++++++++-- > drivers/pci/pci-sysfs.c | 3 +- > drivers/pci/pci.h | 3 +- > include/linux/pci.h | 8 ++ > 5 files changed, 137 insertions(+), 8 deletions(-) > > diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci > index 25c9c39770c6..ebabd0d2ae88 100644 > --- a/Documentation/ABI/testing/sysfs-bus-pci > +++ b/Documentation/ABI/testing/sysfs-bus-pci > @@ -375,3 +375,32 @@ Description: > The value comes from the PCI kernel device state and can be one > of: "unknown", "error", "D0", D1", "D2", "D3hot", "D3cold". > The file is read only. > + > +What: /sys/bus/pci/devices/.../sriov_vf_total_msix > +Date: January 2021 > +Contact: Leon Romanovsky <leonro@xxxxxxxxxx> > +Description: > + This file is associated with a SR-IOV PF. It contains the > + total number of MSI-X vectors available for assignment to > + all VFs associated with PF. It will zero if the device "The value will be zero if the device..." And definition of "VF" and PF" are where in this file? Otherwise looks semi-sane to me, thanks. greg k-h
