On Tue, 19 Mar 2019 10:38:42 +0100 Pierre Morel <pmorel@xxxxxxxxxxxxx> wrote: > On 15/03/2019 19:15, Halil Pasic wrote: > > On Wed, 13 Mar 2019 17:05:01 +0100 > > Pierre Morel <pmorel@xxxxxxxxxxxxx> wrote: > > > >> When the mediated device is open we setup the relation with KVM unset it > >> when the mediated device is released. > >> > >> We ensure KVM is present on opening of the mediated device. > >> > >> We ensure that KVM survives the mediated device, and establish a direct > > > > survives? > > what alternative do you prefer? > Increase kvm's refcount to ensure the guest is alive when the ap_matrix_mdev is active. An ap mp_matrix becomes active with a successful open() and ceases to be active with a release(). Your sentence was materially wrong as the mdev is allowed to outlive the KVM. BTW survive tends to have an 'in spite of' note to it, which outlive does not. vfio-ap is, I hope, not a calamity that threatens the life of KVM ;). https://en.oxforddictionaries.com/definition/survive > > > >> link from KVM to the mediated device to simplify the relationship. > >> > >> Signed-off-by: Pierre Morel <pmorel@xxxxxxxxxxxxx> > >> --- > > ...snip... > > >> static int vfio_ap_mdev_group_notifier(struct notifier_block *nb, > >> unsigned long action, void *data) > >> { > >> - int ret; > >> struct ap_matrix_mdev *matrix_mdev; > >> > >> if (action != VFIO_GROUP_NOTIFY_SET_KVM) > >> return NOTIFY_OK; > >> > >> matrix_mdev = container_of(nb, struct ap_matrix_mdev, group_notifier); > >> - > >> - if (!data) { > >> - matrix_mdev->kvm = NULL; > >> - return NOTIFY_OK; > >> - } > >> - > >> - ret = vfio_ap_mdev_set_kvm(matrix_mdev, data); > >> - if (ret) > >> - return NOTIFY_DONE; > >> - > >> - /* If there is no CRYCB pointer, then we can't copy the masks */ > >> - if (!matrix_mdev->kvm->arch.crypto.crycbd) > >> - return NOTIFY_DONE; > >> - > >> - kvm_arch_crypto_set_masks(matrix_mdev->kvm, matrix_mdev->matrix.apm, > >> - matrix_mdev->matrix.aqm, > >> - matrix_mdev->matrix.adm); > >> + matrix_mdev->kvm = data; > >> > >> return NOTIFY_OK; > >> } > >> @@ -888,6 +873,12 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev) > >> if (ret) > >> goto err_group; > >> > >> + /* We do not support opening the mediated device without KVM */ > >> + if (!matrix_mdev->kvm) { > >> + ret = -ENODEV; > >> + goto err_group; > >> + } > >> + > >> matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier; > >> events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; > >> > >> @@ -896,8 +887,15 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev) > >> if (ret) > >> goto err_iommu; > >> > >> + ret = vfio_ap_mdev_set_kvm(matrix_mdev); > > > > At this point the matrix_mdev->kvm ain't guaranteed to be valid IMHO. Or > > am I wrong? If I'm right kvm_get_kvm(matrix_mdev->kvm) could be too late. > > What about the if (!matrix_mdev->kvm) 10 lines above ? > That check is not sufficient. You should do the kvm_get_kvm() in vfio_ap_mdev_group_notifier(). VFIO must ensure that the kvm pointer you get is valid, in a sense that it points to a valid struct kvm and the kvm object is alive, while you are in the callback. But not beyond. If another thread were to decrement the refcount of the kvm object you would end up with matrix_mdev->kvm pointing to an object that has already died. Does my analysis make sense to you? > > > >> + if (ret) > >> + goto err_kvm; > >> + > >> return 0; > >> > >> +err_kvm: > >> + vfio_unregister_notifier(mdev_dev(mdev), VFIO_IOMMU_NOTIFY, > >> + &matrix_mdev->iommu_notifier); > >> err_iommu: > >> vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, > >> &matrix_mdev->group_notifier); > >> @@ -906,19 +904,33 @@ static int vfio_ap_mdev_open(struct mdev_device *mdev) > >> return ret; > >> } > >> > >> -static void vfio_ap_mdev_release(struct mdev_device *mdev) > >> +static int vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev) > >> { > >> - struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); > >> + struct kvm *kvm = matrix_mdev->kvm; > >> > >> if (matrix_mdev->kvm) > >> kvm_arch_crypto_clear_masks(matrix_mdev->kvm); > > > > This still conditional? > > Yes, nothing to clear if there is no KVM. > Since we have ensured the open only works if there is a KVM at that point in time, and we have taken a reference to KVM, I would expect KVM can not go away before we give up our reference. > > > >> - > >> + vfio_ap_mdev_reset_queues(matrix_mdev->mdev); > > > > I guess your intention was to move vfio_ap_mdev_reset_queues() > > here from vfio_ap_mdev_release(), but you still have a > > vfio_ap_mdev_reset_queues() call in vfio_ap_mdev_release(). > > > >> + matrix_mdev->kvm->arch.crypto.pqap_hook = NULL; > >> matrix_mdev->kvm = NULL; > >> + > >> + kvm_put_kvm(kvm); > >> + return 0; > >> +} > >> + > >> +static void vfio_ap_mdev_release(struct mdev_device *mdev) > >> +{ > >> + struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); > >> + > >> + mutex_lock(&matrix_dev->lock); > >> + > >> vfio_ap_mdev_reset_queues(mdev); > > right, this one will go away. > Thanks for reviewing. > yw Regards, Halil
