On Fri, Nov 18, 2016 at 02:38:47PM +1100, Gavin Shan wrote:
> On Thu, Nov 17, 2016 at 05:30:31PM -0600, Bjorn Helgaas wrote:
> >From: Noa Osherovich <noaos@xxxxxxxxxxxx>
> >
> >Mellanox devices were marked as having INTx masking ability broken. As a
> >result, the VFIO driver fails to start when more than one device function
> >is passed-through to a VM if both have the same INTx pin.
> >
> >Prior to Connect-IB, Mellanox devices exposed to the operating system one
> >PCI function per all ports. Starting from Connect-IB, the devices are
> >function-per-port. When passing the second function to a VM, VFIO will
> >fail to start.
> >
> >Exclude ConnectX-4, ConnectX4-Lx and Connect-IB from the list of Mellanox
> >devices marked as having broken INTx masking:
> >
> >- ConnectX-4 and ConnectX4-LX firmware version is checked. If INTx
> > masking is supported, we unmark the broken INTx masking.
> >- Connect-IB does not support INTx currently so will not cause any
> > problem.
> >
> >[bhelgaas: call pci_disable_device() always, after iounmap()]
> >Fixes: 11e42532ada3 ("PCI: Assume all Mellanox devices have broken INTx masking")
> >Signed-off-by: Noa Osherovich <noaos@xxxxxxxxxxxx>
> >Signed-off-by: Bjorn Helgaas <bhelgaas@xxxxxxxxxx>
> >Reviewed-by: Or Gerlitz <ogerlitz@xxxxxxxxxxxx>
> >Reviewed-by: Gavin Shan <gwshan@xxxxxxxxxxxxxxxxxx>
> >---
> > drivers/pci/quirks.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++---
> > 1 file changed, 54 insertions(+), 3 deletions(-)
> >
> >diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> >index f0515b7..496ca13 100644
> >--- a/drivers/pci/quirks.c
> >+++ b/drivers/pci/quirks.c
> >@@ -3209,13 +3209,25 @@ static u16 mellanox_broken_intx_devs[] = {
> > PCI_DEVICE_ID_MELLANOX_CONNECTX2,
> > PCI_DEVICE_ID_MELLANOX_CONNECTX3,
> > PCI_DEVICE_ID_MELLANOX_CONNECTX3_PRO,
> >- PCI_DEVICE_ID_MELLANOX_CONNECTIB,
> >- PCI_DEVICE_ID_MELLANOX_CONNECTX4,
> >- PCI_DEVICE_ID_MELLANOX_CONNECTX4_LX,
> > };
> >
> >+#define CONNECTX_4_CURR_MAX_MINOR 99
> >+#define CONNECTX_4_INTX_SUPPORT_MINOR 14
> >+
> >+/*
> >+ * Check ConnectX-4/LX FW version to see if it supports legacy interrupts.
> >+ * If so, don't mark it as broken.
> >+ * FW minor > 99 means older FW version format and no INTx masking support.
> >+ * FW minor < 14 means new FW version format and no INTx masking support.
> >+ */
> > static void mellanox_check_broken_intx_masking(struct pci_dev *pdev)
> > {
> >+ __be32 __iomem *fw_ver;
> >+ u16 fw_major;
> >+ u16 fw_minor;
> >+ u16 fw_subminor;
> >+ u32 fw_maj_min;
> >+ u32 fw_sub_min;
> > int i;
> >
> > for (i = 0; i < ARRAY_SIZE(mellanox_broken_intx_devs); i++) {
> >@@ -3224,6 +3236,45 @@ static void mellanox_check_broken_intx_masking(struct pci_dev *pdev)
> > return;
> > }
> > }
> >+
> >+ /* Getting here means Connect-IB cards and up. Connect-IB has no INTx
> >+ * support so shouldn't be checked further
> >+ */
> >+ if (pdev->device == PCI_DEVICE_ID_MELLANOX_CONNECTIB)
> >+ return;
> >+
> >+ if (pdev->device != PCI_DEVICE_ID_MELLANOX_CONNECTX4 &&
> >+ pdev->device != PCI_DEVICE_ID_MELLANOX_CONNECTX4_LX)
> >+ return;
> >+
> >+ /* For ConnectX-4 and ConnectX-4LX, need to check FW support */
> >+ if (pci_enable_device_mem(pdev)) {
> >+ dev_warn(&pdev->dev, "Can't enable device memory\n");
> >+ return;
> >+ }
> >+
> >+ fw_ver = ioremap(pci_resource_start(pdev, 0), 4);
> >+ if (!fw_ver) {
> >+ dev_warn(&pdev->dev, "Can't map ConnectX-4 initialization segment\n");
>
> Bjorn, pci_disable_device() is still missed here. Could you please
> amend it?
Fixed and re-pushed, thanks!
> >+ return;
> >+ }
> >+
> >+ /* Reading from resource space should be 32b aligned */
> >+ fw_maj_min = ioread32be(fw_ver);
> >+ fw_sub_min = ioread32be(fw_ver + 1);
> >+ fw_major = fw_maj_min & 0xffff;
> >+ fw_minor = fw_maj_min >> 16;
> >+ fw_subminor = fw_sub_min & 0xffff;
> >+ if (fw_minor > CONNECTX_4_CURR_MAX_MINOR ||
> >+ fw_minor < CONNECTX_4_INTX_SUPPORT_MINOR) {
> >+ dev_warn(&pdev->dev, "ConnectX-4: FW %u.%u.%u doesn't support INTx masking, disabling. Please upgrade FW to %d.14.1100 and up for INTx support\n",
> >+ fw_major, fw_minor, fw_subminor, pdev->device ==
> >+ PCI_DEVICE_ID_MELLANOX_CONNECTX4 ? 12 : 14);
> >+ pdev->broken_intx_masking = 1;
> >+ }
> >+
> >+ iounmap(fw_ver);
> >+ pci_disable_device(pdev);
> > }
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_MELLANOX, PCI_ANY_ID,
> > mellanox_check_broken_intx_masking);
> >
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
