On Thu, May 27, 2021 at 07:58:12AM +0000, Tian, Kevin wrote: > /dev/ioasid provides an unified interface for managing I/O page tables for > devices assigned to userspace. Device passthrough frameworks (VFIO, vDPA, > etc.) are expected to use this interface instead of creating their own logic to > isolate untrusted device DMAs initiated by userspace. > > This proposal describes the uAPI of /dev/ioasid and also sample sequences > with VFIO as example in typical usages. The driver-facing kernel API provided > by the iommu layer is still TBD, which can be discussed after consensus is > made on this uAPI. > > It's based on a lengthy discussion starting from here: > https://lore.kernel.org/linux-iommu/20210330132830.GO2356281@xxxxxxxxxx/ > > It ends up to be a long writing due to many things to be summarized and > non-trivial effort required to connect them into a complete proposal. > Hope it provides a clean base to converge. Thanks for the writeup. I'm giving this a first pass review, note that I haven't read all the existing replies in detail yet. > > TOC > ==== > 1. Terminologies and Concepts > 2. uAPI Proposal > 2.1. /dev/ioasid uAPI > 2.2. /dev/vfio uAPI > 2.3. /dev/kvm uAPI > 3. Sample structures and helper functions > 4. PASID virtualization > 5. Use Cases and Flows > 5.1. A simple example > 5.2. Multiple IOASIDs (no nesting) > 5.3. IOASID nesting (software) > 5.4. IOASID nesting (hardware) > 5.5. Guest SVA (vSVA) > 5.6. I/O page fault > 5.7. BIND_PASID_TABLE > ==== > > 1. Terminologies and Concepts > ----------------------------------------- > > IOASID FD is the container holding multiple I/O address spaces. User > manages those address spaces through FD operations. Multiple FD's are > allowed per process, but with this proposal one FD should be sufficient for > all intended usages. > > IOASID is the FD-local software handle representing an I/O address space. > Each IOASID is associated with a single I/O page table. IOASIDs can be > nested together, implying the output address from one I/O page table > (represented by child IOASID) must be further translated by another I/O > page table (represented by parent IOASID). Is there a compelling reason to have all the IOASIDs handled by one FD? Simply on the grounds that handles to kernel internal objects are usualy fds, having an fd per ioasid seems like an obvious alternative. In that case plain open() would replace IOASID_ALLOC. Nested could be handled either by 1) having a CREATED_NESTED on the parent fd which spawns a new fd or 2) opening /dev/ioasid again for a new fd and doing a SET_PARENT before doing anything else. I may be bikeshedding here.. > I/O address space can be managed through two protocols, according to > whether the corresponding I/O page table is constructed by the kernel or > the user. When kernel-managed, a dma mapping protocol (similar to > existing VFIO iommu type1) is provided for the user to explicitly specify > how the I/O address space is mapped. Otherwise, a different protocol is > provided for the user to bind an user-managed I/O page table to the > IOMMU, plus necessary commands for iotlb invalidation and I/O fault > handling. > > Pgtable binding protocol can be used only on the child IOASID's, implying > IOASID nesting must be enabled. This is because the kernel doesn't trust > userspace. Nesting allows the kernel to enforce its DMA isolation policy > through the parent IOASID. To clarify, I'm guessing that's a restriction of likely practice, rather than a fundamental API restriction. I can see a couple of theoretical future cases where a user-managed pagetable for a "base" IOASID would be feasible: 1) On some fancy future MMU allowing free nesting, where the kernel would insert an implicit extra layer translating user addresses to physical addresses, and the userspace manages a pagetable with its own VAs being the target AS 2) For a purely software virtual device, where its virtual DMA engine can interpet user addresses fine > IOASID nesting can be implemented in two ways: hardware nesting and > software nesting. With hardware support the child and parent I/O page > tables are walked consecutively by the IOMMU to form a nested translation. > When it's implemented in software, the ioasid driver is responsible for > merging the two-level mappings into a single-level shadow I/O page table. > Software nesting requires both child/parent page tables operated through > the dma mapping protocol, so any change in either level can be captured > by the kernel to update the corresponding shadow mapping. As Jason also said, I don't think you need to restrict software nesting to only kernel managed L2 tables - you already need hooks for cache invalidation, and you can use those to trigger reshadows. > An I/O address space takes effect in the IOMMU only after it is attached > to a device. The device in the /dev/ioasid context always refers to a > physical one or 'pdev' (PF or VF). What you mean by "physical" device here isn't really clear - VFs aren't really physical devices, and the PF/VF terminology also doesn't extent to non-PCI devices (which I think we want to consider for the API, even if we're not implemenenting it any time soon). Now, it's clear that we can't program things into the IOMMU before attaching a device - we might not even know which IOMMU to use. However, I'm not sure if its wise to automatically make the AS "real" as soon as we attach a device: * If we're going to attach a whole bunch of devices, could we (for at least some IOMMU models) end up doing a lot of work which then has to be re-done for each extra device we attach? * With kernel managed IO page tables could attaching a second device (at least on some IOMMU models) require some operation which would require discarding those tables? e.g. if the second device somehow forces a different IO page size For that reason I wonder if we want some sort of explicit enable or activate call. Device attaches would only be valid before, map or attach pagetable calls would only be valid after. > One I/O address space could be attached to multiple devices. In this case, > /dev/ioasid uAPI applies to all attached devices under the specified IOASID. > > Based on the underlying IOMMU capability one device might be allowed > to attach to multiple I/O address spaces, with DMAs accessing them by > carrying different routing information. One of them is the default I/O > address space routed by PCI Requestor ID (RID) or ARM Stream ID. The > remaining are routed by RID + Process Address Space ID (PASID) or > Stream+Substream ID. For simplicity the following context uses RID and > PASID when talking about the routing information for I/O address spaces. I'm not really clear on how this interacts with nested ioasids. Would you generally expect the RID+PASID IOASes to be children of the base RID IOAS, or not? If the PASID ASes are children of the RID AS, can we consider this not as the device explicitly attaching to multiple IOASIDs, but instead attaching to the parent IOASID with awareness of the child ones? > Device attachment is initiated through passthrough framework uAPI (use > VFIO for simplicity in following context). VFIO is responsible for identifying > the routing information and registering it to the ioasid driver when calling > ioasid attach helper function. It could be RID if the assigned device is > pdev (PF/VF) or RID+PASID if the device is mediated (mdev). In addition, > user might also provide its view of virtual routing information (vPASID) in > the attach call, e.g. when multiple user-managed I/O address spaces are > attached to the vfio_device. In this case VFIO must figure out whether > vPASID should be directly used (for pdev) or converted to a kernel- > allocated one (pPASID, for mdev) for physical routing (see section 4). > > Device must be bound to an IOASID FD before attach operation can be > conducted. This is also through VFIO uAPI. In this proposal one device > should not be bound to multiple FD's. Not sure about the gain of > allowing it except adding unnecessary complexity. But if others have > different view we can further discuss. > > VFIO must ensure its device composes DMAs with the routing information > attached to the IOASID. For pdev it naturally happens since vPASID is > directly programmed to the device by guest software. For mdev this > implies any guest operation carrying a vPASID on this device must be > trapped into VFIO and then converted to pPASID before sent to the > device. A detail explanation about PASID virtualization policies can be > found in section 4. > > Modern devices may support a scalable workload submission interface > based on PCI DMWr capability, allowing a single work queue to access > multiple I/O address spaces. One example is Intel ENQCMD, having > PASID saved in the CPU MSR and carried in the instruction payload > when sent out to the device. Then a single work queue shared by > multiple processes can compose DMAs carrying different PASIDs. Is the assumption here that the processes share the IOASID FD instance, but not memory? > When executing ENQCMD in the guest, the CPU MSR includes a vPASID > which, if targeting a mdev, must be converted to pPASID before sent > to the wire. Intel CPU provides a hardware PASID translation capability > for auto-conversion in the fast path. The user is expected to setup the > PASID mapping through KVM uAPI, with information about {vpasid, > ioasid_fd, ioasid}. The ioasid driver provides helper function for KVM > to figure out the actual pPASID given an IOASID. > > With above design /dev/ioasid uAPI is all about I/O address spaces. > It doesn't include any device routing information, which is only > indirectly registered to the ioasid driver through VFIO uAPI. For > example, I/O page fault is always reported to userspace per IOASID, > although it's physically reported per device (RID+PASID). If there is a > need of further relaying this fault into the guest, the user is responsible > of identifying the device attached to this IOASID (randomly pick one if > multiple attached devices) and then generates a per-device virtual I/O > page fault into guest. Similarly the iotlb invalidation uAPI describes the > granularity in the I/O address space (all, or a range), different from the > underlying IOMMU semantics (domain-wide, PASID-wide, range-based). > > I/O page tables routed through PASID are installed in a per-RID PASID > table structure. Some platforms implement the PASID table in the guest > physical space (GPA), expecting it managed by the guest. The guest > PASID table is bound to the IOMMU also by attaching to an IOASID, > representing the per-RID vPASID space. Do we need to consider two management modes here, much as we have for the pagetables themsleves: either kernel managed, in which we have explicit calls to bind a vPASID to a parent PASID, or user managed in which case we register a table in some format. > We propose the host kernel needs to explicitly track guest I/O page > tables even on these platforms, i.e. the same pgtable binding protocol > should be used universally on all platforms (with only difference on who > actually writes the PASID table). One opinion from previous discussion > was treating this special IOASID as a container for all guest I/O page > tables i.e. hiding them from the host. However this way significantly > violates the philosophy in this /dev/ioasid proposal. It is not one IOASID > one address space any more. Device routing information (indirectly > marking hidden I/O spaces) has to be carried in iotlb invalidation and > page faulting uAPI to help connect vIOMMU with the underlying > pIOMMU. This is one design choice to be confirmed with ARM guys. > > Devices may sit behind IOMMU's with incompatible capabilities. The > difference may lie in the I/O page table format, or availability of an user > visible uAPI (e.g. hardware nesting). /dev/ioasid is responsible for > checking the incompatibility between newly-attached device and existing > devices under the specific IOASID and, if found, returning error to user. > Upon such error the user should create a new IOASID for the incompatible > device. > > There is no explicit group enforcement in /dev/ioasid uAPI, due to no > device notation in this interface as aforementioned. But the ioasid driver > does implicit check to make sure that devices within an iommu group > must be all attached to the same IOASID before this IOASID starts to > accept any uAPI command. Otherwise error information is returned to > the user. An explicit ENABLE call might make this checking simpler. > There was a long debate in previous discussion whether VFIO should keep > explicit container/group semantics in its uAPI. Jason Gunthorpe proposes > a simplified model where every device bound to VFIO is explicitly listed > under /dev/vfio thus a device fd can be acquired w/o going through legacy > container/group interface. In this case the user is responsible for > understanding the group topology and meeting the implicit group check > criteria enforced in /dev/ioasid. The use case examples in this proposal > are based on the new model. > > Of course for backward compatibility VFIO still needs to keep the existing > uAPI and vfio iommu type1 will become a shim layer connecting VFIO > iommu ops to internal ioasid helper functions. > > Notes: > - It might be confusing as IOASID is also used in the kernel (drivers/ > iommu/ioasid.c) to represent PCI PASID or ARM substream ID. We need > find a better name later to differentiate. > > - PPC has not be considered yet as we haven't got time to fully understand > its semantics. According to previous discussion there is some generality > between PPC window-based scheme and VFIO type1 semantics. Let's > first make consensus on this proposal and then further discuss how to > extend it to cover PPC's requirement. From what I've seen so far, it seems ok to me. Note that at this stage I'm only familiar with existing PPC IOMMUs, which don't have PASID or anything similar. I'm not sure what IBM's future plans are for IOMMUs, so there will be more checking to be done. > - There is a protocol between vfio group and kvm. Needs to think about > how it will be affected following this proposal. I think that's only used on PPC, as an optimization for PAPR's paravirt IOMMU with a small default IOVA window. I think we can do something equivalent for IOASIDs from what I've seen so far. > - mdev in this context refers to mediated subfunctions (e.g. Intel SIOV) > which can be physically isolated in-between through PASID-granular > IOMMU protection. Historically people also discussed one usage by > mediating a pdev into a mdev. This usage is not covered here, and is > supposed to be replaced by Max's work which allows overriding various > VFIO operations in vfio-pci driver. I think there are a couple of different mdev cases, so we'll need to be careful of that and clarify our terminology a bit, I think. > 2. uAPI Proposal > ---------------------- > > /dev/ioasid uAPI covers everything about managing I/O address spaces. > > /dev/vfio uAPI builds connection between devices and I/O address spaces. > > /dev/kvm uAPI is optional required as far as ENQCMD is concerned. > > > 2.1. /dev/ioasid uAPI > +++++++++++++++++ > > /* > * Check whether an uAPI extension is supported. > * > * This is for FD-level capabilities, such as locked page pre-registration. > * IOASID-level capabilities are reported through IOASID_GET_INFO. > * > * Return: 0 if not supported, 1 if supported. > */ > #define IOASID_CHECK_EXTENSION _IO(IOASID_TYPE, IOASID_BASE + 0) > > > /* > * Register user space memory where DMA is allowed. > * > * It pins user pages and does the locked memory accounting so sub- > * sequent IOASID_MAP/UNMAP_DMA calls get faster. > * > * When this ioctl is not used, one user page might be accounted > * multiple times when it is mapped by multiple IOASIDs which are > * not nested together. > * > * Input parameters: > * - vaddr; > * - size; > * > * Return: 0 on success, -errno on failure. > */ > #define IOASID_REGISTER_MEMORY _IO(IOASID_TYPE, IOASID_BASE + 1) > #define IOASID_UNREGISTER_MEMORY _IO(IOASID_TYPE, IOASID_BASE + 2) AIUI PPC is the main user of the current pre-registration API, though it could have value in any vIOMMU case to avoid possibly costly accounting on every guest map/unmap. I wonder if there's a way to model this using a nested AS rather than requiring special operations. e.g. 'prereg' IOAS | \- 'rid' IOAS | \- 'pasid' IOAS (maybe) 'prereg' would have a kernel managed pagetable into which (for example) qemu platform code would map all guest memory (using IOASID_MAP_DMA). qemu's vIOMMU driver would then mirror the guest's IO mappings into the 'rid' IOAS in terms of GPA. This wouldn't quite work as is, because the 'prereg' IOAS would have no devices. But we could potentially have another call to mark an IOAS as a purely "preregistration" or pure virtual IOAS. Using that would be an alternative to attaching devices. > /* > * Allocate an IOASID. > * > * IOASID is the FD-local software handle representing an I/O address > * space. Each IOASID is associated with a single I/O page table. User > * must call this ioctl to get an IOASID for every I/O address space that is > * intended to be enabled in the IOMMU. > * > * A newly-created IOASID doesn't accept any command before it is > * attached to a device. Once attached, an empty I/O page table is > * bound with the IOMMU then the user could use either DMA mapping > * or pgtable binding commands to manage this I/O page table. > * > * Device attachment is initiated through device driver uAPI (e.g. VFIO) > * > * Return: allocated ioasid on success, -errno on failure. > */ > #define IOASID_ALLOC _IO(IOASID_TYPE, IOASID_BASE + 3) > #define IOASID_FREE _IO(IOASID_TYPE, IOASID_BASE + 4) > > > /* > * Get information about an I/O address space > * > * Supported capabilities: > * - VFIO type1 map/unmap; > * - pgtable/pasid_table binding > * - hardware nesting vs. software nesting; > * - ... > * > * Related attributes: > * - supported page sizes, reserved IOVA ranges (DMA mapping); Can I request we represent this in terms of permitted IOVA ranges, rather than reserved IOVA ranges. This works better with the "window" model I have in mind for unifying the restrictions of the POWER IOMMU with Type1 like mapping. > * - vendor pgtable formats (pgtable binding); > * - number of child IOASIDs (nesting); > * - ... > * > * Above information is available only after one or more devices are > * attached to the specified IOASID. Otherwise the IOASID is just a > * number w/o any capability or attribute. > * > * Input parameters: > * - u32 ioasid; > * > * Output parameters: > * - many. TBD. > */ > #define IOASID_GET_INFO _IO(IOASID_TYPE, IOASID_BASE + 5) > > > /* > * Map/unmap process virtual addresses to I/O virtual addresses. > * > * Provide VFIO type1 equivalent semantics. Start with the same > * restriction e.g. the unmap size should match those used in the > * original mapping call. > * > * If IOASID_REGISTER_MEMORY has been called, the mapped vaddr > * must be already in the preregistered list. > * > * Input parameters: > * - u32 ioasid; > * - refer to vfio_iommu_type1_dma_{un}map > * > * Return: 0 on success, -errno on failure. > */ > #define IOASID_MAP_DMA _IO(IOASID_TYPE, IOASID_BASE + 6) > #define IOASID_UNMAP_DMA _IO(IOASID_TYPE, IOASID_BASE + 7) I'm assuming these would be expected to fail if a user managed pagetable has been bound? > /* > * Create a nesting IOASID (child) on an existing IOASID (parent) > * > * IOASIDs can be nested together, implying that the output address > * from one I/O page table (child) must be further translated by > * another I/O page table (parent). > * > * As the child adds essentially another reference to the I/O page table > * represented by the parent, any device attached to the child ioasid > * must be already attached to the parent. > * > * In concept there is no limit on the number of the nesting levels. > * However for the majority case one nesting level is sufficient. The > * user should check whether an IOASID supports nesting through > * IOASID_GET_INFO. For example, if only one nesting level is allowed, > * the nesting capability is reported only on the parent instead of the > * child. > * > * User also needs check (via IOASID_GET_INFO) whether the nesting > * is implemented in hardware or software. If software-based, DMA > * mapping protocol should be used on the child IOASID. Otherwise, > * the child should be operated with pgtable binding protocol. > * > * Input parameters: > * - u32 parent_ioasid; > * > * Return: child_ioasid on success, -errno on failure; > */ > #define IOASID_CREATE_NESTING _IO(IOASID_TYPE, IOASID_BASE + 8) > > > /* > * Bind an user-managed I/O page table with the IOMMU > * > * Because user page table is untrusted, IOASID nesting must be enabled > * for this ioasid so the kernel can enforce its DMA isolation policy > * through the parent ioasid. > * > * Pgtable binding protocol is different from DMA mapping. The latter > * has the I/O page table constructed by the kernel and updated > * according to user MAP/UNMAP commands. With pgtable binding the > * whole page table is created and updated by userspace, thus different > * set of commands are required (bind, iotlb invalidation, page fault, etc.). > * > * Because the page table is directly walked by the IOMMU, the user > * must use a format compatible to the underlying hardware. It can > * check the format information through IOASID_GET_INFO. > * > * The page table is bound to the IOMMU according to the routing > * information of each attached device under the specified IOASID. The > * routing information (RID and optional PASID) is registered when a > * device is attached to this IOASID through VFIO uAPI. > * > * Input parameters: > * - child_ioasid; > * - address of the user page table; > * - formats (vendor, address_width, etc.); > * > * Return: 0 on success, -errno on failure. > */ > #define IOASID_BIND_PGTABLE _IO(IOASID_TYPE, IOASID_BASE + 9) > #define IOASID_UNBIND_PGTABLE _IO(IOASID_TYPE, IOASID_BASE + 10) I'm assuming that UNBIND would return the IOASID to a kernel-managed pagetable? For debugging and certain hypervisor edge cases it might be useful to have a call to allow userspace to lookup and specific IOVA in a guest managed pgtable. > /* > * Bind an user-managed PASID table to the IOMMU > * > * This is required for platforms which place PASID table in the GPA space. > * In this case the specified IOASID represents the per-RID PASID space. > * > * Alternatively this may be replaced by IOASID_BIND_PGTABLE plus a > * special flag to indicate the difference from normal I/O address spaces. > * > * The format info of the PASID table is reported in IOASID_GET_INFO. > * > * As explained in the design section, user-managed I/O page tables must > * be explicitly bound to the kernel even on these platforms. It allows > * the kernel to uniformly manage I/O address spaces cross all platforms. > * Otherwise, the iotlb invalidation and page faulting uAPI must be hacked > * to carry device routing information to indirectly mark the hidden I/O > * address spaces. > * > * Input parameters: > * - child_ioasid; Wouldn't this be the parent ioasid, rather than one of the potentially many child ioasids? > * - address of PASID table; > * - formats (vendor, size, etc.); > * > * Return: 0 on success, -errno on failure. > */ > #define IOASID_BIND_PASID_TABLE _IO(IOASID_TYPE, IOASID_BASE + 11) > #define IOASID_UNBIND_PASID_TABLE _IO(IOASID_TYPE, IOASID_BASE + 12) > > > /* > * Invalidate IOTLB for an user-managed I/O page table > * > * Unlike what's defined in include/uapi/linux/iommu.h, this command > * doesn't allow the user to specify cache type and likely support only > * two granularities (all, or a specified range) in the I/O address space. > * > * Physical IOMMU have three cache types (iotlb, dev_iotlb and pasid > * cache). If the IOASID represents an I/O address space, the invalidation > * always applies to the iotlb (and dev_iotlb if enabled). If the IOASID > * represents a vPASID space, then this command applies to the PASID > * cache. > * > * Similarly this command doesn't provide IOMMU-like granularity > * info (domain-wide, pasid-wide, range-based), since it's all about the > * I/O address space itself. The ioasid driver walks the attached > * routing information to match the IOMMU semantics under the > * hood. > * > * Input parameters: > * - child_ioasid; And couldn't this be be any ioasid, not just a child one, depending on whether you want PASID scope or RID scope invalidation? > * - granularity > * > * Return: 0 on success, -errno on failure > */ > #define IOASID_INVALIDATE_CACHE _IO(IOASID_TYPE, IOASID_BASE + 13) > > > /* > * Page fault report and response > * > * This is TBD. Can be added after other parts are cleared up. Likely it > * will be a ring buffer shared between user/kernel, an eventfd to notify > * the user and an ioctl to complete the fault. > * > * The fault data is per I/O address space, i.e.: IOASID + faulting_addr > */ > > > /* > * Dirty page tracking > * > * Track and report memory pages dirtied in I/O address spaces. There > * is an ongoing work by Kunkun Jiang by extending existing VFIO type1. > * It needs be adapted to /dev/ioasid later. > */ > > > 2.2. /dev/vfio uAPI > ++++++++++++++++ > > /* > * Bind a vfio_device to the specified IOASID fd > * > * Multiple vfio devices can be bound to a single ioasid_fd, but a single > * vfio device should not be bound to multiple ioasid_fd's. > * > * Input parameters: > * - ioasid_fd; > * > * Return: 0 on success, -errno on failure. > */ > #define VFIO_BIND_IOASID_FD _IO(VFIO_TYPE, VFIO_BASE + 22) > #define VFIO_UNBIND_IOASID_FD _IO(VFIO_TYPE, VFIO_BASE + 23) > > > /* > * Attach a vfio device to the specified IOASID > * > * Multiple vfio devices can be attached to the same IOASID, and vice > * versa. > * > * User may optionally provide a "virtual PASID" to mark an I/O page > * table on this vfio device. Whether the virtual PASID is physically used > * or converted to another kernel-allocated PASID is a policy in vfio device > * driver. > * > * There is no need to specify ioasid_fd in this call due to the assumption > * of 1:1 connection between vfio device and the bound fd. > * > * Input parameter: > * - ioasid; > * - flag; > * - user_pasid (if specified); Wouldn't the PASID be communicated by whether you give a parent or child ioasid, rather than needing an extra value? > * Return: 0 on success, -errno on failure. > */ > #define VFIO_ATTACH_IOASID _IO(VFIO_TYPE, VFIO_BASE + 24) > #define VFIO_DETACH_IOASID _IO(VFIO_TYPE, VFIO_BASE + 25) > > > 2.3. KVM uAPI > ++++++++++++ > > /* > * Update CPU PASID mapping > * > * This is necessary when ENQCMD will be used in the guest while the > * targeted device doesn't accept the vPASID saved in the CPU MSR. > * > * This command allows user to set/clear the vPASID->pPASID mapping > * in the CPU, by providing the IOASID (and FD) information representing > * the I/O address space marked by this vPASID. > * > * Input parameters: > * - user_pasid; > * - ioasid_fd; > * - ioasid; > */ > #define KVM_MAP_PASID _IO(KVMIO, 0xf0) > #define KVM_UNMAP_PASID _IO(KVMIO, 0xf1) > > > 3. Sample structures and helper functions > -------------------------------------------------------- > > Three helper functions are provided to support VFIO_BIND_IOASID_FD: > > struct ioasid_ctx *ioasid_ctx_fdget(int fd); > int ioasid_register_device(struct ioasid_ctx *ctx, struct ioasid_dev *dev); > int ioasid_unregister_device(struct ioasid_dev *dev); > > An ioasid_ctx is created for each fd: > > struct ioasid_ctx { > // a list of allocated IOASID data's > struct list_head ioasid_list; > // a list of registered devices > struct list_head dev_list; > // a list of pre-registered virtual address ranges > struct list_head prereg_list; > }; > > Each registered device is represented by ioasid_dev: > > struct ioasid_dev { > struct list_head next; > struct ioasid_ctx *ctx; > // always be the physical device Again "physical" isn't really clearly defined here. > struct device *device; > struct kref kref; > }; > > Because we assume one vfio_device connected to at most one ioasid_fd, > here ioasid_dev could be embedded in vfio_device and then linked to > ioasid_ctx->dev_list when registration succeeds. For mdev the struct > device should be the pointer to the parent device. PASID marking this > mdev is specified later when VFIO_ATTACH_IOASID. > > An ioasid_data is created when IOASID_ALLOC, as the main object > describing characteristics about an I/O page table: > > struct ioasid_data { > // link to ioasid_ctx->ioasid_list > struct list_head next; > > // the IOASID number > u32 ioasid; > > // the handle to convey iommu operations > // hold the pgd (TBD until discussing iommu api) > struct iommu_domain *domain; > > // map metadata (vfio type1 semantics) > struct rb_node dma_list; Why do you need this? Can't you just store the kernel managed mappings in the host IO pgtable? > // pointer to user-managed pgtable (for nesting case) > u64 user_pgd; > // link to the parent ioasid (for nesting) > struct ioasid_data *parent; > > // cache the global PASID shared by ENQCMD-capable > // devices (see below explanation in section 4) > u32 pasid; > > // a list of device attach data (routing information) > struct list_head attach_data; > > // a list of partially-attached devices (group) > struct list_head partial_devices; > > // a list of fault_data reported from the iommu layer > struct list_head fault_data; > > ... > } > > ioasid_data and iommu_domain have overlapping roles as both are > introduced to represent an I/O address space. It is still a big TBD how > the two should be corelated or even merged, and whether new iommu > ops are required to handle RID+PASID explicitly. We leave this as open > for now as this proposal is mainly about uAPI. For simplification > purpose the two objects are kept separate in this context, assuming an > 1:1 connection in-between and the domain as the place-holder > representing the 1st class object in the iommu ops. > > Two helper functions are provided to support VFIO_ATTACH_IOASID: > > struct attach_info { > u32 ioasid; > // If valid, the PASID to be used physically > u32 pasid; Again shouldn't the choice of a parent or child ioasid inform whether there is a pasid, and if so which one? > }; > int ioasid_device_attach(struct ioasid_dev *dev, > struct attach_info info); > int ioasid_device_detach(struct ioasid_dev *dev, u32 ioasid); > > The pasid parameter is optionally provided based on the policy in vfio > device driver. It could be the PASID marking the default I/O address > space for a mdev, or the user-provided PASID marking an user I/O page > table, or another kernel-allocated PASID backing the user-provided one. > Please check next section for detail explanation. > > A new object is introduced and linked to ioasid_data->attach_data for > each successful attach operation: > > struct ioasid_attach_data { > struct list_head next; > struct ioasid_dev *dev; > u32 pasid; > } > > As explained in the design section, there is no explicit group enforcement > in /dev/ioasid uAPI or helper functions. But the ioasid driver does > implicit group check - before every device within an iommu group is > attached to this IOASID, the previously-attached devices in this group are > put in ioasid_data->partial_devices. The IOASID rejects any command if > the partial_devices list is not empty. > > Then is the last helper function: > u32 ioasid_get_global_pasid(struct ioasid_ctx *ctx, > u32 ioasid, bool alloc); > > ioasid_get_global_pasid is necessary in scenarios where multiple devices > want to share a same PASID value on the attached I/O page table (e.g. > when ENQCMD is enabled, as explained in next section). We need a > centralized place (ioasid_data->pasid) to hold this value (allocated when > first called with alloc=true). vfio device driver calls this function (alloc= > true) to get the global PASID for an ioasid before calling ioasid_device_ > attach. KVM also calls this function (alloc=false) to setup PASID translation > structure when user calls KVM_MAP_PASID. > > 4. PASID Virtualization > ------------------------------ > > When guest SVA (vSVA) is enabled, multiple GVA address spaces are > created on the assigned vfio device. This leads to the concepts of > "virtual PASID" (vPASID) vs. "physical PASID" (pPASID). vPASID is assigned > by the guest to mark an GVA address space while pPASID is the one > selected by the host and actually routed in the wire. > > vPASID is conveyed to the kernel when user calls VFIO_ATTACH_IOASID. > > vfio device driver translates vPASID to pPASID before calling ioasid_attach_ > device, with two factors to be considered: > > - Whether vPASID is directly used (vPASID==pPASID) in the wire, or > should be instead converted to a newly-allocated one (vPASID!= > pPASID); > > - If vPASID!=pPASID, whether pPASID is allocated from per-RID PASID > space or a global PASID space (implying sharing pPASID cross devices, > e.g. when supporting Intel ENQCMD which puts PASID in a CPU MSR > as part of the process context); > > The actual policy depends on pdev vs. mdev, and whether ENQCMD is > supported. There are three possible scenarios: > > (Note: /dev/ioasid uAPI is not affected by underlying PASID virtualization > policies.) > > 1) pdev (w/ or w/o ENQCMD): vPASID==pPASID > > vPASIDs are directly programmed by the guest to the assigned MMIO > bar, implying all DMAs out of this device having vPASID in the packet > header. This mandates vPASID==pPASID, sort of delegating the entire > per-RID PASID space to the guest. > > When ENQCMD is enabled, the CPU MSR when running a guest task > contains a vPASID. In this case the CPU PASID translation capability > should be disabled so this vPASID in CPU MSR is directly sent to the > wire. > > This ensures consistent vPASID usage on pdev regardless of the > workload submitted through a MMIO register or ENQCMD instruction. > > 2) mdev: vPASID!=pPASID (per-RID if w/o ENQCMD, otherwise global) > > PASIDs are also used by kernel to mark the default I/O address space > for mdev, thus cannot be delegated to the guest. Instead, the mdev > driver must allocate a new pPASID for each vPASID (thus vPASID!= > pPASID) and then use pPASID when attaching this mdev to an ioasid. > > The mdev driver needs cache the PASID mapping so in mediation > path vPASID programmed by the guest can be converted to pPASID > before updating the physical MMIO register. The mapping should > also be saved in the CPU PASID translation structure (via KVM uAPI), > so the vPASID saved in the CPU MSR is auto-translated to pPASID > before sent to the wire, when ENQCMD is enabled. > > Generally pPASID could be allocated from the per-RID PASID space > if all mdev's created on the parent device don't support ENQCMD. > > However if the parent supports ENQCMD-capable mdev, pPASIDs > must be allocated from a global pool because the CPU PASID > translation structure is per-VM. It implies that when an guest I/O > page table is attached to two mdevs with a single vPASID (i.e. bind > to the same guest process), a same pPASID should be used for > both mdevs even when they belong to different parents. Sharing > pPASID cross mdevs is achieved by calling aforementioned ioasid_ > get_global_pasid(). > > 3) Mix pdev/mdev together > > Above policies are per device type thus are not affected when mixing > those device types together (when assigned to a single guest). However, > there is one exception - when both pdev/mdev support ENQCMD. > > Remember the two types have conflicting requirements on whether > CPU PASID translation should be enabled. This capability is per-VM, > and must be enabled for mdev isolation. When enabled, pdev will > receive a mdev pPASID violating its vPASID expectation. > > In previous thread a PASID range split scheme was discussed to support > this combination, but we haven't worked out a clean uAPI design yet. > Therefore in this proposal we decide to not support it, implying the > user should have some intelligence to avoid such scenario. It could be > a TODO task for future. > > In spite of those subtle considerations, the kernel implementation could > start simple, e.g.: > > - v==p for pdev; > - v!=p and always use a global PASID pool for all mdev's; > > Regardless of the kernel policy, the user policy is unchanged: > > - provide vPASID when calling VFIO_ATTACH_IOASID; > - call KVM uAPI to setup CPU PASID translation if ENQCMD-capable mdev; > - Don't expose ENQCMD capability on both pdev and mdev; > > Sample user flow is described in section 5.5. > > 5. Use Cases and Flows > ------------------------------- > > Here assume VFIO will support a new model where every bound device > is explicitly listed under /dev/vfio thus a device fd can be acquired w/o > going through legacy container/group interface. For illustration purpose > those devices are just called dev[1...N]: > > device_fd[1...N] = open("/dev/vfio/devices/dev[1...N]", mode); Minor detail, but I'd suggest /dev/vfio/pci/DDDD:BB:SS.F for the filenames for actual PCI functions. Maybe /dev/vfio/mdev/something for mdevs. That leaves other subdirs of /dev/vfio free for future non-PCI device types, and /dev/vfio itself for the legacy group devices. > As explained earlier, one IOASID fd is sufficient for all intended use cases: > > ioasid_fd = open("/dev/ioasid", mode); > > For simplicity below examples are all made for the virtualization story. > They are representative and could be easily adapted to a non-virtualization > scenario. > > Three types of IOASIDs are considered: > > gpa_ioasid[1...N]: for GPA address space > giova_ioasid[1...N]: for guest IOVA address space > gva_ioasid[1...N]: for guest CPU VA address space > > At least one gpa_ioasid must always be created per guest, while the other > two are relevant as far as vIOMMU is concerned. > > Examples here apply to both pdev and mdev, if not explicitly marked out > (e.g. in section 5.5). VFIO device driver in the kernel will figure out the > associated routing information in the attaching operation. > > For illustration simplicity, IOASID_CHECK_EXTENSION and IOASID_GET_ > INFO are skipped in these examples. > > 5.1. A simple example > ++++++++++++++++++ > > Dev1 is assigned to the guest. One gpa_ioasid is created. The GPA address > space is managed through DMA mapping protocol: > > /* Bind device to IOASID fd */ > device_fd = open("/dev/vfio/devices/dev1", mode); > ioasid_fd = open("/dev/ioasid", mode); > ioctl(device_fd, VFIO_BIND_IOASID_FD, ioasid_fd); > > /* Attach device to IOASID */ > gpa_ioasid = ioctl(ioasid_fd, IOASID_ALLOC); > at_data = { .ioasid = gpa_ioasid}; > ioctl(device_fd, VFIO_ATTACH_IOASID, &at_data); > > /* Setup GPA mapping */ > dma_map = { > .ioasid = gpa_ioasid; > .iova = 0; // GPA > .vaddr = 0x40000000; // HVA > .size = 1GB; > }; > ioctl(ioasid_fd, IOASID_DMA_MAP, &dma_map); > > If the guest is assigned with more than dev1, user follows above sequence > to attach other devices to the same gpa_ioasid i.e. sharing the GPA > address space cross all assigned devices. > > 5.2. Multiple IOASIDs (no nesting) > ++++++++++++++++++++++++++++ > > Dev1 and dev2 are assigned to the guest. vIOMMU is enabled. Initially > both devices are attached to gpa_ioasid. Doesn't really affect your example, but note that the PAPR IOMMU does not have a passthrough mode, so devices will not initially be attached to gpa_ioasid - they will be unusable for DMA until attached to a gIOVA ioasid. > After boot the guest creates > an GIOVA address space (giova_ioasid) for dev2, leaving dev1 in pass > through mode (gpa_ioasid). > > Suppose IOASID nesting is not supported in this case. Qemu need to > generate shadow mappings in userspace for giova_ioasid (like how > VFIO works today). > > To avoid duplicated locked page accounting, it's recommended to pre- > register the virtual address range that will be used for DMA: > > device_fd1 = open("/dev/vfio/devices/dev1", mode); > device_fd2 = open("/dev/vfio/devices/dev2", mode); > ioasid_fd = open("/dev/ioasid", mode); > ioctl(device_fd1, VFIO_BIND_IOASID_FD, ioasid_fd); > ioctl(device_fd2, VFIO_BIND_IOASID_FD, ioasid_fd); > > /* pre-register the virtual address range for accounting */ > mem_info = { .vaddr = 0x40000000; .size = 1GB }; > ioctl(ioasid_fd, IOASID_REGISTER_MEMORY, &mem_info); > /* Attach dev1 and dev2 to gpa_ioasid */ > gpa_ioasid = ioctl(ioasid_fd, IOASID_ALLOC); > at_data = { .ioasid = gpa_ioasid}; > ioctl(device_fd1, VFIO_ATTACH_IOASID, &at_data); > ioctl(device_fd2, VFIO_ATTACH_IOASID, &at_data); > > /* Setup GPA mapping */ > dma_map = { > .ioasid = gpa_ioasid; > .iova = 0; // GPA > .vaddr = 0x40000000; // HVA > .size = 1GB; > }; > ioctl(ioasid_fd, IOASID_DMA_MAP, &dma_map); > > /* After boot, guest enables an GIOVA space for dev2 */ Again, doesn't break the example, but this need not happen after guest boot. On the PAPR vIOMMU, the guest IOVA spaces (known as "logical IO bus numbers" / liobns) and which devices are in each are fixed at guest creation time and advertised to the guest via firmware. > giova_ioasid = ioctl(ioasid_fd, IOASID_ALLOC); > > /* First detach dev2 from previous address space */ > at_data = { .ioasid = gpa_ioasid}; > ioctl(device_fd2, VFIO_DETACH_IOASID, &at_data); > > /* Then attach dev2 to the new address space */ > at_data = { .ioasid = giova_ioasid}; > ioctl(device_fd2, VFIO_ATTACH_IOASID, &at_data); > > /* Setup a shadow DMA mapping according to vIOMMU > * GIOVA (0x2000) -> GPA (0x1000) -> HVA (0x40001000) > */ > dma_map = { > .ioasid = giova_ioasid; > .iova = 0x2000; // GIOVA > .vaddr = 0x40001000; // HVA > .size = 4KB; > }; > ioctl(ioasid_fd, IOASID_DMA_MAP, &dma_map); > > 5.3. IOASID nesting (software) > +++++++++++++++++++++++++ > > Same usage scenario as 5.2, with software-based IOASID nesting > available. In this mode it is the kernel instead of user to create the > shadow mapping. In this case, I feel like the preregistration is redundant with the GPA level mapping. As long as the gIOVA mappings (which might be frequent) can piggyback on the accounting done for the GPA mapping we accomplish what we need from preregistration. > The flow before guest boots is same as 5.2, except one point. Because > giova_ioasid is nested on gpa_ioasid, locked accounting is only > conducted for gpa_ioasid. So it's not necessary to pre-register virtual > memory. > > To save space we only list the steps after boots (i.e. both dev1/dev2 > have been attached to gpa_ioasid before guest boots): > > /* After boots */ > /* Make GIOVA space nested on GPA space */ > giova_ioasid = ioctl(ioasid_fd, IOASID_CREATE_NESTING, > gpa_ioasid); > > /* Attach dev2 to the new address space (child) > * Note dev2 is still attached to gpa_ioasid (parent) > */ > at_data = { .ioasid = giova_ioasid}; > ioctl(device_fd2, VFIO_ATTACH_IOASID, &at_data); > > /* Setup a GIOVA->GPA mapping for giova_ioasid, which will be > * merged by the kernel with GPA->HVA mapping of gpa_ioasid > * to form a shadow mapping. > */ > dma_map = { > .ioasid = giova_ioasid; > .iova = 0x2000; // GIOVA > .vaddr = 0x1000; // GPA > .size = 4KB; > }; > ioctl(ioasid_fd, IOASID_DMA_MAP, &dma_map); > > 5.4. IOASID nesting (hardware) > +++++++++++++++++++++++++ > > Same usage scenario as 5.2, with hardware-based IOASID nesting > available. In this mode the pgtable binding protocol is used to > bind the guest IOVA page table with the IOMMU: > > /* After boots */ > /* Make GIOVA space nested on GPA space */ > giova_ioasid = ioctl(ioasid_fd, IOASID_CREATE_NESTING, > gpa_ioasid); > > /* Attach dev2 to the new address space (child) > * Note dev2 is still attached to gpa_ioasid (parent) > */ > at_data = { .ioasid = giova_ioasid}; > ioctl(device_fd2, VFIO_ATTACH_IOASID, &at_data); > > /* Bind guest I/O page table */ > bind_data = { > .ioasid = giova_ioasid; > .addr = giova_pgtable; > // and format information > }; > ioctl(ioasid_fd, IOASID_BIND_PGTABLE, &bind_data); > > /* Invalidate IOTLB when required */ > inv_data = { > .ioasid = giova_ioasid; > // granular information > }; > ioctl(ioasid_fd, IOASID_INVALIDATE_CACHE, &inv_data); > > /* See 5.6 for I/O page fault handling */ > > 5.5. Guest SVA (vSVA) > ++++++++++++++++++ > > After boots the guest further create a GVA address spaces (gpasid1) on > dev1. Dev2 is not affected (still attached to giova_ioasid). > > As explained in section 4, user should avoid expose ENQCMD on both > pdev and mdev. > > The sequence applies to all device types (being pdev or mdev), except > one additional step to call KVM for ENQCMD-capable mdev: > > /* After boots */ > /* Make GVA space nested on GPA space */ > gva_ioasid = ioctl(ioasid_fd, IOASID_CREATE_NESTING, > gpa_ioasid); I'm not clear what gva_ioasid is representing. Is it representing a single vPASID's address space, or a whole bunch of vPASIDs address spaces? > /* Attach dev1 to the new address space and specify vPASID */ > at_data = { > .ioasid = gva_ioasid; > .flag = IOASID_ATTACH_USER_PASID; > .user_pasid = gpasid1; > }; > ioctl(device_fd1, VFIO_ATTACH_IOASID, &at_data); > > /* if dev1 is ENQCMD-capable mdev, update CPU PASID > * translation structure through KVM > */ > pa_data = { > .ioasid_fd = ioasid_fd; > .ioasid = gva_ioasid; > .guest_pasid = gpasid1; > }; > ioctl(kvm_fd, KVM_MAP_PASID, &pa_data); > > /* Bind guest I/O page table */ > bind_data = { > .ioasid = gva_ioasid; > .addr = gva_pgtable1; > // and format information > }; > ioctl(ioasid_fd, IOASID_BIND_PGTABLE, &bind_data); > > ... > > > 5.6. I/O page fault > +++++++++++++++ > > (uAPI is TBD. Here is just about the high-level flow from host IOMMU driver > to guest IOMMU driver and backwards). > > - Host IOMMU driver receives a page request with raw fault_data {rid, > pasid, addr}; > > - Host IOMMU driver identifies the faulting I/O page table according to > information registered by IOASID fault handler; > > - IOASID fault handler is called with raw fault_data (rid, pasid, addr), which > is saved in ioasid_data->fault_data (used for response); > > - IOASID fault handler generates an user fault_data (ioasid, addr), links it > to the shared ring buffer and triggers eventfd to userspace; > > - Upon received event, Qemu needs to find the virtual routing information > (v_rid + v_pasid) of the device attached to the faulting ioasid. If there are > multiple, pick a random one. This should be fine since the purpose is to > fix the I/O page table on the guest; > > - Qemu generates a virtual I/O page fault through vIOMMU into guest, > carrying the virtual fault data (v_rid, v_pasid, addr); > > - Guest IOMMU driver fixes up the fault, updates the I/O page table, and > then sends a page response with virtual completion data (v_rid, v_pasid, > response_code) to vIOMMU; > > - Qemu finds the pending fault event, converts virtual completion data > into (ioasid, response_code), and then calls a /dev/ioasid ioctl to > complete the pending fault; > > - /dev/ioasid finds out the pending fault data {rid, pasid, addr} saved in > ioasid_data->fault_data, and then calls iommu api to complete it with > {rid, pasid, response_code}; > > 5.7. BIND_PASID_TABLE > ++++++++++++++++++++ > > PASID table is put in the GPA space on some platform, thus must be updated > by the guest. It is treated as another user page table to be bound with the > IOMMU. > > As explained earlier, the user still needs to explicitly bind every user I/O > page table to the kernel so the same pgtable binding protocol (bind, cache > invalidate and fault handling) is unified cross platforms. > > vIOMMUs may include a caching mode (or paravirtualized way) which, once > enabled, requires the guest to invalidate PASID cache for any change on the > PASID table. This allows Qemu to track the lifespan of guest I/O page tables. > > In case of missing such capability, Qemu could enable write-protection on > the guest PASID table to achieve the same effect. > > /* After boots */ > /* Make vPASID space nested on GPA space */ > pasidtbl_ioasid = ioctl(ioasid_fd, IOASID_CREATE_NESTING, > gpa_ioasid); I think this time pasidtbl_ioasid is representing multiple vPASID address spaces, yes? In which case I don't think it should be treated as the same sort of object as a normal IOASID, which represents a single address space IIUC. > /* Attach dev1 to pasidtbl_ioasid */ > at_data = { .ioasid = pasidtbl_ioasid}; > ioctl(device_fd1, VFIO_ATTACH_IOASID, &at_data); > > /* Bind PASID table */ > bind_data = { > .ioasid = pasidtbl_ioasid; > .addr = gpa_pasid_table; > // and format information > }; > ioctl(ioasid_fd, IOASID_BIND_PASID_TABLE, &bind_data); > > /* vIOMMU detects a new GVA I/O space created */ > gva_ioasid = ioctl(ioasid_fd, IOASID_CREATE_NESTING, > gpa_ioasid); > > /* Attach dev1 to the new address space, with gpasid1 */ > at_data = { > .ioasid = gva_ioasid; > .flag = IOASID_ATTACH_USER_PASID; > .user_pasid = gpasid1; > }; > ioctl(device_fd1, VFIO_ATTACH_IOASID, &at_data); > > /* Bind guest I/O page table. Because SET_PASID_TABLE has been > * used, the kernel will not update the PASID table. Instead, just > * track the bound I/O page table for handling invalidation and > * I/O page faults. > */ > bind_data = { > .ioasid = gva_ioasid; > .addr = gva_pgtable1; > // and format information > }; > ioctl(ioasid_fd, IOASID_BIND_PGTABLE, &bind_data); Hrm.. if you still have to individually bind a table for each vPASID, what's the point of BIND_PASID_TABLE? -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Attachment:
signature.asc
Description: PGP signature
