The callback functions for probing and removing a queue device must take
and release the locks required to perform a dynamic update of a guest's
APCB in the proper order.
The proper order for taking the locks is:
matrix_dev->guests_lock => kvm->lock => matrix_dev->mdevs_lock
The proper order for releasing the locks is:
matrix_dev->mdevs_lock => kvm->lock => matrix_dev->guests_lock
A new helper function is introduced to be used by the probe callback to
acquire the required locks. Since the probe callback only has
access to a queue device when it is called, the helper function will
find
the ap_matrix_mdev object to which the queue device's APQN is
assigned and
return it so the KVM guest to which the mdev is attached can be
dynamically
updated.
Note that in order to find the ap_matrix_mdev (matrix_mdev) object,
it is
necessary to search the matrix_dev->mdev_list. This presents a
locking order dilemma because the matrix_dev->mdevs_lock can't be
taken to
protect against changes to the list while searching for the
matrix_mdev to
which a queue device's APQN is assigned. This is due to the fact that
the
proper locking order requires that the matrix_dev->mdevs_lock be taken
after both the matrix_mdev->kvm->lock and the matrix_dev->mdevs_lock.
Consequently, the matrix_dev->guests_lock will be used to protect
against
removal of a matrix_mdev object from the list while a queue device is
being probed. This necessitates changes to the mdev probe/remove
callback functions to take the matrix_dev->guests_lock prior to removing
a matrix_mdev object from the list.
A new macro is also introduced to acquire the locks required to
dynamically
update the guest's APCB in the proper order when a queue device is
removed.
Signed-off-by: Tony Krowiak <akrowiak@xxxxxxxxxxxxx>
---
drivers/s390/crypto/vfio_ap_ops.c | 126 +++++++++++++++++++++---------
1 file changed, 88 insertions(+), 38 deletions(-)
diff --git a/drivers/s390/crypto/vfio_ap_ops.c
b/drivers/s390/crypto/vfio_ap_ops.c
index 2219b1069ceb..080a733f7cd2 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -116,6 +116,74 @@ static const struct vfio_device_ops
vfio_ap_matrix_dev_ops;
mutex_unlock(&matrix_dev->guests_lock); \
})
+/**
+ * vfio_ap_mdev_get_update_locks_for_apqn: retrieve the matrix mdev
to which an
+ * APQN is assigned and acquire the
+ * locks required to update the APCB of
+ * the KVM guest to which the mdev is
+ * attached.
+ *
+ * @apqn: the APQN of a queue device.
+ *
+ * The proper locking order is:
+ * 1. matrix_dev->guests_lock: required to use the KVM pointer to
update a KVM
+ * guest's APCB.
+ * 2. matrix_mdev->kvm->lock: required to update a guest's APCB
+ * 3. matrix_dev->mdevs_lock: required to access data stored in a
matrix_mdev
+ *
+ * Note: If @apqn is not assigned to a matrix_mdev, the
matrix_mdev->kvm->lock
+ * will not be taken.
+ *
+ * Return: the ap_matrix_mdev object to which @apqn is assigned or
NULL if @apqn
+ * is not assigned to an ap_matrix_mdev.
+ */
+static struct ap_matrix_mdev
*vfio_ap_mdev_get_update_locks_for_apqn(int apqn)
+{
+ struct ap_matrix_mdev *matrix_mdev;
+
+ mutex_lock(&matrix_dev->guests_lock);
+
+ list_for_each_entry(matrix_mdev, &matrix_dev->mdev_list, node) {
+ if (test_bit_inv(AP_QID_CARD(apqn), matrix_mdev->matrix.apm) &&
+ test_bit_inv(AP_QID_QUEUE(apqn),
matrix_mdev->matrix.aqm)) {
+ if (matrix_mdev->kvm)
+ mutex_lock(&matrix_mdev->kvm->lock);
+
+ mutex_lock(&matrix_dev->mdevs_lock);
+
+ return matrix_mdev;
+ }
+ }
+
+ mutex_lock(&matrix_dev->mdevs_lock);
+
+ return NULL;
+}
+
+/**
+ * get_update_locks_for_queue: get the locks required to update the
APCB of the
+ * KVM guest to which the matrix mdev linked to a
+ * vfio_ap_queue object is attached.
+ *
+ * @queue: a pointer to a vfio_ap_queue object.
+ *
+ * The proper locking order is:
+ * 1. matrix_dev->guests_lock: required to use the KVM pointer to
update a KVM
+ * guest's APCB.
+ * 2. queue->matrix_mdev->kvm->lock: required to update a guest's APCB
+ * 3. matrix_dev->mdevs_lock: required to access data stored in a
matrix_mdev
+ *
+ * Note: if @queue is not linked to an ap_matrix_mdev object, the
KVM lock
+ * will not be taken.
+ */
+#define get_update_locks_for_queue(queue) ({ \
+ struct ap_matrix_mdev *matrix_mdev = q->matrix_mdev; \
+ mutex_lock(&matrix_dev->guests_lock); \
+ if (matrix_mdev && matrix_mdev->kvm) \
+ mutex_lock(&matrix_mdev->kvm->lock); \
+ mutex_lock(&matrix_dev->mdevs_lock); \
+})
+
