> From: Jason Gunthorpe via iommu
> Sent: Friday, December 10, 2021 12:08 AM
>
> On Thu, Dec 09, 2021 at 03:59:57AM +0000, Tian, Kevin wrote:
> > > From: Tian, Kevin
> > > Sent: Thursday, December 9, 2021 10:58 AM
> > >
> > > For ARM it's SMMU's PASID table format. There is no step-2 since PASID
> > > is already within the address space covered by the user PASID table.
> > >
> >
> > One correction here. 'no step-2' is definitely wrong here as it means
> > more than user page table in your plan (e.g. dpdk).
> >
> > To simplify it what I meant is:
> >
> > iommufd reports how many 'user page tables' are supported given a device.
> >
> > ARM always reports only one can be supported, and it must be created in
> > PASID table format. tagged by RID.
> >
> > Intel reports one in step1 (tagged by RID), and N in step2 (tagged by
> > RID+PASID). A special flag in attach call allows the user to specify the
> > additional PASID routing info for a 'user page table'.
>
> I don't think 'number of user page tables' makes sense
>
> It really is 'attach to the whole device' vs 'attach to the RID' as a
> semantic that should exist
>
> If we imagine a userspace using kernel page tables it certainly makes
> sense to assign page table A to the RID and page table B to a PASID
> even in simple cases like vfio-pci.
>
> The only case where userspace would want to capture the entire RID and
> all PASIDs is something like this ARM situation - but userspace just
> created a device specific object and already knows exactly what kind
> of behavior it has.
>
> So, something like vfio pci would implement three uAPI operations:
> - Attach page table to RID
> - Attach page table to PASID
> - Attach page table to RID and all PASIDs
> And here 'page table' is everything below the STE in SMMUv3
>
> While mdev can only support:
> - Access emulated page table
> - Attach page table to PASID
mdev is a pci device from user p.o.v, having its vRID and vPASID. From
this angle the uAPI is no different from vfio-pci (except the ARM one):
- (sw mdev) Attach emulated page table to vRID (no iommu domain)
- (hw mdev) Attach page table to vRID (mapped to mdev PASID)
- (hw mdev) Attach page table to vPASID (mapped to a fungible PASID)
>
> It is what I've said a couple of times, the API the driver calls
> toward iommufd to attach a page table must be unambiguous as to the
> intention, which also means userspace must be unambiguous too.
>
No question on the unambiguous part. But we also need to consider
the common semantics that can be abstracted.
>From user p.o.v a vRID can be attached to at most two page tables (if
nesting is enabled). This just requires the basic attaching form for
either one page table or two page tables:
at_data = {
.iommufd = xxx;
.pgtable_id = yyy;
};
ioctl(device_fd, VFIO_DEVICE_ATTACH_PGTABLE, &at_data);
This can already cover ARM's requirement. The user page table
attached to vRID is in vendor specific format, e.g. either ARM pasid
table format or Intel stage-1 format. For ARM pasid_table + underlying
stage-1 page tables can be considered as a single big paging structure.
>From this angle I'm not sure the benefit of making a separate uAPI
just because it's a pasid table for ARM.
Then when PASID needs to be explicitly specified (e.g. in Intel case):
at_data = {
.iommufd = xxx;
.pgtable_id = yyy;
.flags = VFIO_ATTACH_FLAGS_PASID;
.pasid = zzz;
};
ioctl(device_fd, VFIO_DEVICE_ATTACH_PGTABLE, &at_data);
Again, I don't think what a simple flag can solve needs to be made
into a separate uAPI.
Is modeling like above considered ambiguous?
Thanks
Kevin
