On Thu, 26 Jul 2018 21:54:25 +0200
Christian Borntraeger <borntraeger@xxxxxxxxxx> wrote:
> From: Tony Krowiak <akrowiak@xxxxxxxxxxxxx>
>
> Let's call PAPQ(ZAPQ) to zeroize a queue:
>
> * For each queue configured for a mediated matrix device
> when it is released.
>
> * When an AP queue is unbound from the VFIO AP device driver.
OK, that confuses me...
>
> Zeroizing a queue resets the queue, clears all pending
> messages for the queue entries and disables adapter interruptions
> associated with the queue.
>
> Signed-off-by: Tony Krowiak <akrowiak@xxxxxxxxxxxxx>
> Reviewed-by: Halil Pasic <pasic@xxxxxxxxxxxxx>
> Tested-by: Michael Mueller <mimu@xxxxxxxxxxxxx>
> Tested-by: Farhan Ali <alifm@xxxxxxxxxxxxx>
> Signed-off-by: Christian Borntraeger <borntraeger@xxxxxxxxxx>
> ---
> drivers/s390/crypto/vfio_ap_drv.c | 6 ++++-
> drivers/s390/crypto/vfio_ap_ops.c | 32 +++++++++++++++++++++++++++
> drivers/s390/crypto/vfio_ap_private.h | 25 +++++++++++++++++++++
> 3 files changed, 62 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/s390/crypto/vfio_ap_drv.c b/drivers/s390/crypto/vfio_ap_drv.c
> index 25636e6bf893..936f025fc8e9 100644
> --- a/drivers/s390/crypto/vfio_ap_drv.c
> +++ b/drivers/s390/crypto/vfio_ap_drv.c
> @@ -59,7 +59,11 @@ static int vfio_ap_queue_dev_probe(struct ap_device *apdev)
>
> static void vfio_ap_queue_dev_remove(struct ap_device *apdev)
> {
> - /* Nothing to do yet */
> + /*
> + * NOTE: We still regard the queue as ours. For taking away queues
> + * from vfio_ap the ap bus will provide a callback interface, so we
> + * can take away the queues from the guests if needed.
> + */
...as we don't seem to do anything here yet?
> }
>
> static int vfio_ap_matrix_dev_create(void)
> diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
> index 01c429cb51d5..461a74515d3d 100644
> --- a/drivers/s390/crypto/vfio_ap_ops.c
> +++ b/drivers/s390/crypto/vfio_ap_ops.c
> @@ -803,6 +803,37 @@ static int vfio_ap_mdev_open_once(struct ap_matrix_mdev *matrix_mdev)
> return ret;
> }
>
> +static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev, bool force)
> +{
> + int ret;
> + int rc = 0;
> + unsigned long apid, apqi;
> + struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
> +
> + if (!matrix_mdev->activated) {
> + pr_err("%s: reset failed for mdev %s, not activated",
> + VFIO_AP_MODULE_NAME, matrix_mdev->name);
> +
> + return -EPERM;
> + }
Hm... if we stick with the activation approach, what happens if we:
- open the mdev
- activate the matrix
- deactivate the matrix
- release the mdev, triggering this function
It seems we won't call PAPQ(ZAPQ) in that case -- is that how it is
supposed to work?
> +
> + for_each_set_bit_inv(apid, matrix_mdev->matrix.apm,
> + matrix_mdev->matrix.apm_max + 1) {
> + for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm,
> + matrix_mdev->matrix.aqm_max + 1) {
> + ret = vfio_ap_reset_queue(apid, apqi, 1);
> + if (ret) {
> + if (force)
> + rc = ret;
> + else
> + return ret;
> + }
> + }
> + }
> +
> + return rc;
> +}
> +
> static int vfio_ap_mdev_open(struct mdev_device *mdev)
> {
> struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev);
> @@ -860,6 +891,7 @@ static void vfio_ap_mdev_release(struct mdev_device *mdev)
>
> mutex_lock(&matrix_dev.lock);
> kvm_arch_crypto_clear_masks(matrix_mdev->kvm);
> + vfio_ap_mdev_reset_queues(mdev, true);
> vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY,
> &matrix_mdev->group_notifier);
> matrix_mdev->kvm = NULL;
> diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h
> index 34be9afe9ced..f11457ca90f6 100644
> --- a/drivers/s390/crypto/vfio_ap_private.h
> +++ b/drivers/s390/crypto/vfio_ap_private.h
> @@ -76,4 +76,29 @@ static inline struct device *to_device(struct ap_matrix_dev *matrix_dev)
> extern int vfio_ap_mdev_register(void);
> extern void vfio_ap_mdev_unregister(void);
>
> +static inline int vfio_ap_reset_queue(unsigned int apid, unsigned int apqi,
> + unsigned int retry)
> +{
> + struct ap_queue_status status;
> +
> + do {
> + status = ap_zapq(AP_MKQID(apid, apqi));
> + switch (status.response_code) {
> + case AP_RESPONSE_NORMAL:
> + return 0;
> + case AP_RESPONSE_RESET_IN_PROGRESS:
> + case AP_RESPONSE_BUSY:
> + msleep(20);
> + break;
> + default:
> + pr_warn("%s: error zeroizing %02x.%04x: response code %d\n",
> + VFIO_AP_MODULE_NAME, apid, apqi,
> + status.response_code);
Again, introducing a dbf may be better than dumping too many messages
into the syslog.
> + return -EIO;
> + }
> + } while (retry--);
> +
> + return -EBUSY;
> +}
> +
> #endif /* _VFIO_AP_PRIVATE_H_ */
