On Tue, Feb 9, 2021 at 5:34 AM Leon Romanovsky <leon@xxxxxxxxxx> wrote: > > From: Leon Romanovsky <leonro@xxxxxxxxxx> <snip> > -------------------------------------------------------------------- > Hi, > > The number of MSI-X vectors is PCI property visible through lspci, that > field is read-only and configured by the device. > > The static assignment of an amount of MSI-X vectors doesn't allow utilize > the newly created VF because it is not known to the device the future load > and configuration where that VF will be used. > > The VFs are created on the hypervisor and forwarded to the VMs that have > different properties (for example number of CPUs). > > To overcome the inefficiency in the spread of such MSI-X vectors, we > allow the kernel to instruct the device with the needed number of such > vectors, before VF is initialized and bounded to the driver. > > Before this series: > [root@server ~]# lspci -vs 0000:08:00.2 > 08:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] > .... > Capabilities: [9c] MSI-X: Enable- Count=12 Masked- > > Configuration script: > 1. Start fresh > echo 0 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs > modprobe -q -r mlx5_ib mlx5_core > 2. Ensure that driver doesn't run and it is safe to change MSI-X > echo 0 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_drivers_autoprobe > 3. Load driver for the PF > modprobe mlx5_core > 4. Configure one of the VFs with new number > echo 2 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs > echo 21 > /sys/bus/pci/devices/0000\:08\:00.2/sriov_vf_msix_count > > After this series: > [root@server ~]# lspci -vs 0000:08:00.2 > 08:00.2 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5 Virtual Function] > .... > Capabilities: [9c] MSI-X: Enable- Count=21 Masked- > > Thanks > > Leon Romanovsky (4): > PCI: Add sysfs callback to allow MSI-X table size change of SR-IOV VFs > net/mlx5: Add dynamic MSI-X capabilities bits > net/mlx5: Dynamically assign MSI-X vectors count > net/mlx5: Allow to the users to configure number of MSI-X vectors > > Documentation/ABI/testing/sysfs-bus-pci | 28 ++++ > .../net/ethernet/mellanox/mlx5/core/main.c | 17 ++ > .../ethernet/mellanox/mlx5/core/mlx5_core.h | 27 ++++ > .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 72 +++++++++ > .../net/ethernet/mellanox/mlx5/core/sriov.c | 58 ++++++- > drivers/pci/iov.c | 153 ++++++++++++++++++ > include/linux/mlx5/mlx5_ifc.h | 11 +- > include/linux/pci.h | 12 ++ > 8 files changed, 375 insertions(+), 3 deletions(-) > This seems much improved from the last time I reviewed the patch set. I am good with the drop of the folder in favor of using "sriov" in the naming of the fields. For the series: Reviewed-by: Alexander Duyck <alexanderduyck@xxxxxx>
