Hi David,
On 6/17/21 1:22 PM, David Gibson wrote:
The iommu_group can guarantee the isolation among different physical
devices (represented by RIDs). But when it comes to sub-devices (ex. mdev or
vDPA devices represented by RID + SSID), we have to rely on the
device driver for isolation. The devices which are able to generate sub-
devices should either use their own on-device mechanisms or use the
platform features like Intel Scalable IOV to isolate the sub-devices.
This seems like a misunderstanding of groups. Groups are not tied to
any PCI meaning. Groups are the smallest unit of isolation, no matter
what is providing that isolation.
If mdevs are isolated from each other by clever software, even though
they're on the same PCI device they are in different groups from each
other*by definition*. They are also in a different group from their
parent device (however the mdevs only exist when mdev driver is
active, which implies that the parent device's group is owned by the
kernel).
You are right. This is also my understanding of an "isolation group".
But, as I understand it, iommu_group is only the isolation group visible
to IOMMU. When we talk about sub-devices (sw-mdev or mdev w/ pasid),
only the device and device driver knows the details of isolation, hence
iommu_group could not be extended to cover them. The device drivers
should define their own isolation groups.
Otherwise, the device driver has to fake an iommu_group and add hacky
code to link the related IOMMU elements (iommu device, domain, group
etc.) together. Actually this is part of the problem that this proposal
tries to solve.
Under above conditions, different sub-device from a same RID device
could be able to use different IOASID. This seems to means that we can't
support mixed mode where, for example, two RIDs share an iommu_group and
one (or both) of them have sub-devices.
That doesn't necessarily follow. mdevs which can be successfully
isolated by their mdev driver are in a different group from their
parent device, and therefore need not be affected by whether the
parent device shares a group with some other physical device. They
*might* be, but that's up to the mdev driver to determine based on
what it can safely isolate.
If we understand it as multiple levels of isolation, can we classify the
devices into the following categories?
1) Legacy devices
- devices without device-level isolation
- multiple devices could sit in a single iommu_group
- only a single I/O address space could be bound to IOMMU
2) Modern devices
- devices capable of device-level isolation
- able to have subdevices
- self-isolated, hence not share iommu_group with others
- multiple I/O address spaces could be bound to IOMMU
For 1), all devices in an iommu_group should be bound to a single
IOASID; The isolation is guaranteed by an iommu_group.
For 2) a single device could be bound to multiple IOASIDs with each sub-
device corresponding to an IOASID. The isolation of each subdevice is
guaranteed by the device driver.
Best regards,
baolu
