On Sat, 13 May 2023 06:28:07 -0700
Yi Liu <yi.l.liu@xxxxxxxxx> wrote:
> This makes the vfio file kAPIs to accept vfio device files, also a
> preparation for vfio device cdev support.
>
> For the kvm set with vfio device file, kvm pointer is stored in struct
> vfio_device_file, and use kvm_ref_lock to protect kvm set and kvm
> pointer usage within VFIO. This kvm pointer will be set to vfio_device
> after device file is bound to iommufd in the cdev path.
>
> Reviewed-by: Kevin Tian <kevin.tian@xxxxxxxxx>
> Reviewed-by: Jason Gunthorpe <jgg@xxxxxxxxxx>
> Tested-by: Terrence Xu <terrence.xu@xxxxxxxxx>
> Tested-by: Nicolin Chen <nicolinc@xxxxxxxxxx>
> Tested-by: Matthew Rosato <mjrosato@xxxxxxxxxxxxx>
> Tested-by: Yanting Jiang <yanting.jiang@xxxxxxxxx>
> Tested-by: Shameer Kolothum <shameerali.kolothum.thodi@xxxxxxxxxx>
> Signed-off-by: Yi Liu <yi.l.liu@xxxxxxxxx>
> ---
> drivers/vfio/vfio.h | 2 ++
> drivers/vfio/vfio_main.c | 36 +++++++++++++++++++++++++++++++++++-
> 2 files changed, 37 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> index b1e327a85a32..69e1a0692b06 100644
> --- a/drivers/vfio/vfio.h
> +++ b/drivers/vfio/vfio.h
> @@ -18,6 +18,8 @@ struct vfio_container;
>
> struct vfio_device_file {
> struct vfio_device *device;
> + spinlock_t kvm_ref_lock; /* protect kvm field */
> + struct kvm *kvm;
> };
>
> void vfio_device_put_registration(struct vfio_device *device);
> diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
> index 4665791aa2eb..8ef9210ad2aa 100644
> --- a/drivers/vfio/vfio_main.c
> +++ b/drivers/vfio/vfio_main.c
> @@ -429,6 +429,7 @@ vfio_allocate_device_file(struct vfio_device *device)
> return ERR_PTR(-ENOMEM);
>
> df->device = device;
> + spin_lock_init(&df->kvm_ref_lock);
>
> return df;
> }
> @@ -1190,13 +1191,23 @@ const struct file_operations vfio_device_fops = {
> .mmap = vfio_device_fops_mmap,
> };
>
> +static struct vfio_device *vfio_device_from_file(struct file *file)
> +{
> + struct vfio_device_file *df = file->private_data;
> +
> + if (file->f_op != &vfio_device_fops)
> + return NULL;
> + return df->device;
> +}
> +
> /**
> * vfio_file_is_valid - True if the file is valid vfio file
> * @file: VFIO group file or VFIO device file
> */
> bool vfio_file_is_valid(struct file *file)
> {
> - return vfio_group_from_file(file);
> + return vfio_group_from_file(file) ||
> + vfio_device_from_file(file);
> }
> EXPORT_SYMBOL_GPL(vfio_file_is_valid);
>
> @@ -1211,16 +1222,36 @@ EXPORT_SYMBOL_GPL(vfio_file_is_valid);
> */
> bool vfio_file_enforced_coherent(struct file *file)
> {
> + struct vfio_device *device;
> struct vfio_group *group;
>
> group = vfio_group_from_file(file);
> if (group)
> return vfio_group_enforced_coherent(group);
>
> + device = vfio_device_from_file(file);
> + if (device)
> + return device_iommu_capable(device->dev,
> + IOMMU_CAP_ENFORCE_CACHE_COHERENCY);
> +
> return true;
> }
> EXPORT_SYMBOL_GPL(vfio_file_enforced_coherent);
>
> +static void vfio_device_file_set_kvm(struct file *file, struct kvm *kvm)
A general nit, we've been trying to maintain function naming based on
the object it operates on in vfio, for example vfio_group_set_kvm()
clearly operates on the struct vfio_group object. Here we have
vfio_device_file_set_kvm(), which would suggest it works on a
struct vfio_device_file, but we're passing a struct file.
vfio_file_set_kvm() is already taken below, so should this be:
static void vfio_df_set_kvm(struct vfio_device_file *df,
struct kvm *kvm)
After this series We end up with a number of functions where the object
doesn't really match, ex:
vfio_device_open -> vfio_df_open
vfio_device_close -> vfio_df_close
vfio_device_group_close -> vfio_df_group_close
vfio_iommufd_bind -> vfio_df_iommufd_bind
vfio_iommufd_unbind -> vfio_df_iommufd_unbind
vfio_device_cdev_close -> vfio_df_cdev_close
vfio_device_ioctl_bind_iommufd -> vfio_df_ioctl_bind_iommufd
vfio_ioctl_device_attach -> vfio_df_ioctl_attach_pt
vfio_ioctl_device_detach -> vfio_df_ioctl_detach_pt
"df" is just a suggestion, maybe someone has a better one. Thanks,
Alex
> +{
> + struct vfio_device_file *df = file->private_data;
> +
> + /*
> + * The kvm is first recorded in the vfio_device_file, and will
> + * be propagated to vfio_device::kvm when the file is bound to
> + * iommufd successfully in the vfio device cdev path.
> + */
> + spin_lock(&df->kvm_ref_lock);
> + df->kvm = kvm;
> + spin_unlock(&df->kvm_ref_lock);
> +}
> +
> /**
> * vfio_file_set_kvm - Link a kvm with VFIO drivers
> * @file: VFIO group file or VFIO device file
> @@ -1236,6 +1267,9 @@ void vfio_file_set_kvm(struct file *file, struct kvm *kvm)
> group = vfio_group_from_file(file);
> if (group)
> vfio_group_set_kvm(group, kvm);
> +
> + if (vfio_device_from_file(file))
> + vfio_device_file_set_kvm(file, kvm);
> }
> EXPORT_SYMBOL_GPL(vfio_file_set_kvm);
>
