From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@xxxxxxxxxxxxxxx>
As per PCIe spec r5.0, sec 9.3.7, in SR-IOV devices, capabilities like
PASID, PRI, VC, etc are shared between PF and its associated VFs. So, to
prevent race conditions between PF/VF while updating configuration
registers of these shared capabilities, a new synchronization mechanism
is required.
As a first step, create shared resource lock and expose expose
pci_physfn_reslock/unlock() API's. Users of these shared capabilities can
use these lock/unlock interfaces to synchronize its access.
Since the shared capability is always implemented by PF, reslock mutex
has been added to pci_sriov structure which only exists for PF.
NOTE: Currently this reslock is common for all shared capabilities
between PF/VF. In future, if any performance impact has been noticed, we
should create individual locks for each of the shared capability.
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@xxxxxxxxxxxxxxx>
---
drivers/pci/iov.c | 1 +
drivers/pci/pci.h | 40 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index 525fd3f272b3..004e7076b065 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -507,6 +507,7 @@ static int sriov_init(struct pci_dev *dev, int pos)
else
iov->dev = dev;
+ mutex_init(&iov->reslock);
dev->sriov = iov;
dev->is_physfn = 1;
rc = compute_max_vf_buses(dev);
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index d22d1b807701..c7fa09f3389a 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -304,6 +304,19 @@ struct pci_sriov {
u16 subsystem_device; /* VF subsystem device */
resource_size_t barsz[PCI_SRIOV_NUM_BARS]; /* VF BAR size */
bool drivers_autoprobe; /* Auto probing of VFs by driver */
+ /*
+ * reslock mutex is used for synchronizing updates to resources
+ * shared between PF and all associated VFs. For example, in
+ * SRIOV devices, PRI and PASID interfaces are shared between
+ * PF an all VFs, and hence we need proper locking mechanism to
+ * prevent both PF and VF update the PRI or PASID configuration
+ * registers at the same time.
+ * NOTE: Currently, this lock is shared by all capabilities that
+ * has shared resource between PF and VFs. If there is any performance
+ * impact then perhaps we need to create separate lock for each of
+ * the independent capability that has shared resources.
+ */
+ struct mutex reslock; /* PF/VF shared resource lock */
};
/**
@@ -433,6 +446,27 @@ void pci_iov_update_resource(struct pci_dev *dev, int resno);
resource_size_t pci_sriov_resource_alignment(struct pci_dev *dev, int resno);
void pci_restore_iov_state(struct pci_dev *dev);
int pci_iov_bus_range(struct pci_bus *bus);
+static inline void pci_physfn_reslock(struct pci_dev *dev)
+{
+ struct pci_dev *pf = pci_physfn(dev);
+
+ /* For non SRIOV devices, locking is not needed */
+ if (!pf->is_physfn)
+ return;
+
+ mutex_lock(&pf->sriov->reslock);
+}
+
+static inline void pci_physfn_resunlock(struct pci_dev *dev)
+{
+ struct pci_dev *pf = pci_physfn(dev);
+
+ /* For non SRIOV devices, reslock is never held */
+ if (!pf->is_physfn)
+ return;
+
+ mutex_unlock(&pf->sriov->reslock);
+}
#else
static inline int pci_iov_init(struct pci_dev *dev)
@@ -453,6 +487,12 @@ static inline int pci_iov_bus_range(struct pci_bus *bus)
{
return 0;
}
+static inline void pci_physfn_reslock(struct pci_dev *dev)
+{
+}
+static inline void pci_physfn_resunlock(struct pci_dev *dev)
+{
+}
#endif /* CONFIG_PCI_IOV */
--
2.21.0
