On Thu, May 30, 2024 at 12:28:43AM +0000, Tian, Kevin wrote: > > From: Nicolin Chen <nicolinc@xxxxxxxxxx> > > Sent: Wednesday, May 29, 2024 11:21 AM > > On Wed, May 29, 2024 at 02:58:11AM +0000, Tian, Kevin wrote: > > > My question is why that option is chosen instead of going with 1:1 > > > mapping between vSMMU and viommu i.e. letting the kernel to > > > figure out which pSMMU should be sent an invalidation cmd to, as > > > how VT-d is virtualized. > > > > > > I want to know whether doing so is simply to be compatible with > > > what VCMDQ requires, or due to another untold reason. > > > > Because we use viommu as a VMID holder for SMMU. So a pSMMU must > > have its own viommu to store its VMID for a shared s2_hwpt: > > |-- viommu0 (VMIDx) --|-- pSMMU0 --| > > vSMMU--|-- viommu1 (VMIDy) --|-- pSMMU1 --|--s2_hwpt > > |-- viommu2 (VMIDz) --|-- pSMMU2 --| > > > > there are other options, e.g. you can have one viommu holding multiple > VMIDs each associating to a pSMMU. Well, possibly. But everything previously in a viommu would have to be a list (for number of VMIDs) in the kernel level: not only a VMID list, but also a 2D virtual ID lists, something like: struct xarray vdev_ids[num_of_vmid]; // per-IOMMU vID to pID lookup And a driver in this case would be difficult to get a complete concept of a viommu object since it's core object and shared by all kernel-level IOMMU instances. If a driver wants to extend a viommu object for some additional feature, e.g. VINTF in this series, it would likely have to create another per-driver object and again another list of this kind of objects in struct viommu. At the end of day, we have to duplicate it one way or another for the amount of physical IOMMUs. And it seems to cleaner by doing it with multiple viommu objects. > so it's really an implementation choice that you want a same viommu > topology scheme w/ or w/o VCMDQ. > > we all know there are some constraints of copying the physical topology, > e.g. hotplugging a device or migration. for VCMDQ it's clearly an > acceptable tradeoff for performance. w/o VCMDQ I'm not sure whether > you want to keep more flexibility for the user. 😊 Oh. With regular nested SMMU, there is only one virtual SMMU in the guest VM. No need of copying physical topology. Just the VMM needs to allocate three viommus to add them to a list of its own. Thanks Nicolin
