> From: Peter Xu [mailto:peterx@xxxxxxxxxx] > Sent: Wednesday, September 25, 2019 3:45 PM > > On Wed, Sep 25, 2019 at 07:21:51AM +0000, Tian, Kevin wrote: > > > From: Peter Xu [mailto:peterx@xxxxxxxxxx] > > > Sent: Wednesday, September 25, 2019 2:57 PM > > > > > > On Wed, Sep 25, 2019 at 10:48:32AM +0800, Lu Baolu wrote: > > > > Hi Kevin, > > > > > > > > On 9/24/19 3:00 PM, Tian, Kevin wrote: > > > > > > > > '-----------' > > > > > > > > '-----------' > > > > > > > > > > > > > > > > This patch series only aims to achieve the first goal, a.k.a using > > > > > first goal? then what are other goals? I didn't spot such information. > > > > > > > > > > > > > The overall goal is to use IOMMU nested mode to avoid shadow page > > > table > > > > and VMEXIT when map an gIOVA. This includes below 4 steps (maybe > not > > > > accurate, but you could get the point.) > > > > > > > > 1) GIOVA mappings over 1st-level page table; > > > > 2) binding vIOMMU 1st level page table to the pIOMMU; > > > > 3) using pIOMMU second level for GPA->HPA translation; > > > > 4) enable nested (a.k.a. dual stage) translation in host. > > > > > > > > This patch set aims to achieve 1). > > > > > > Would it make sense to use 1st level even for bare-metal to replace > > > the 2nd level? > > > > > > What I'm thinking is the DPDK apps - they have MMU page table already > > > there for the huge pages, then if they can use 1st level as the > > > default device page table then it even does not need to map, because > > > it can simply bind the process root page table pointer to the 1st > > > level page root pointer of the device contexts that it uses. > > > > > > > Then you need bear with possible page faults from using CPU page > > table, while most devices don't support it today. > > Right, I was just thinking aloud. After all neither do we have IOMMU > hardware to support 1st level (or am I wrong?)... It's just that when You are right. Current VT-d supports only 2nd level. > the 1st level is ready it should sound doable because IIUC PRI should > be always with the 1st level support no matter on IOMMU side or the > device side? No. PRI is not tied to 1st or 2nd level. Actually from device p.o.v, it's just a protocol to trigger page fault, but the device doesn't care whether the page fault is on 1st or 2nd level in the IOMMU side. The only relevant part is that a PRI request can have PASID tagged or cleared. When it's tagged with PASID, the IOMMU will locate the translation table under the given PASID (either 1st or 2nd level is fine, according to PASID entry setting). When no PASID is included, the IOMMU locates the translation from default entry (e.g. PASID#0 or any PASID contained in RID2PASID in VT-d). Your knowledge happened to be correct in deprecated ECS mode. At that time, there is only one 2nd level per context entry which doesn't support page fault, and there is only one 1st level per PASID entry which supports page fault. Then PRI could be indirectly connected to 1st level, but this just changed with new scalable mode. Another note is that the PRI capability only indicates that a device is capable of handling page faults, but not that a device can tolerate page fault for any of its DMA access. If the latter is fasle, using CPU page table for DPDK usage is still risky (and specific to device behavior) > > I'm actually not sure about whether my understanding here is > correct... I thought the pasid binding previously was only for some > vendor kernel drivers but not a general thing to userspace. I feel > like that should be doable in the future once we've got some new > syscall interface ready to deliver 1st level page table (e.g., via > vfio?) then applications like DPDK seems to be able to use that too > even directly via bare metal. > using 1st level for userspace is different from supporting DMA page fault in userspace. The former is purely about which structure to keep the mapping. I think we may do the same thing for both bare metal and guest (using 2nd level only for GPA when nested is enabled on the IOMMU). But reusing CPU page table for userspace is more tricky. :-) Thanks Kevin
