On Mon, Oct 23, 2017 at 06:37:29PM +0800, Kenneth Lee wrote: > Date: Mon, 23 Oct 2017 18:37:29 +0800 > From: Kenneth Lee <liguozhu@xxxxxxxxxxxxx> > To: Jean-Philippe Brucker <jean-philippe.brucker@xxxxxxx> > CC: Alex Williamson <alex.williamson@xxxxxxxxxx>, > "kenneth-lee-2012@xxxxxxxxxxx" <kenneth-lee-2012@xxxxxxxxxxx>, Jon Masters > <jcm@xxxxxxxxxxxxxx>, "jcm@xxxxxxxxxx" <jcm@xxxxxxxxxx>, > "xuzaibo@xxxxxxxxxx" <xuzaibo@xxxxxxxxxx>, > "linux-accelerators@xxxxxxxxxxxxxxxx" > <linux-accelerators@xxxxxxxxxxxxxxxx> > Subject: Re: To extend the feature of vfio-mdev > Message-ID: <20171023103729.GF211370@Turing-Arch-b> > > On Mon, Oct 23, 2017 at 09:31:25AM +0100, Jean-Philippe Brucker wrote: > > Date: Mon, 23 Oct 2017 09:31:25 +0100 > > From: Jean-Philippe Brucker <jean-philippe.brucker@xxxxxxx> > > To: Kenneth Lee <liguozhu@xxxxxxxxxxxxx>, Alex Williamson > > <alex.williamson@xxxxxxxxxx> > > CC: "kenneth-lee-2012@xxxxxxxxxxx" <kenneth-lee-2012@xxxxxxxxxxx>, Jon > > Masters <jcm@xxxxxxxxxxxxxx>, "jcm@xxxxxxxxxx" <jcm@xxxxxxxxxx>, > > "xuzaibo@xxxxxxxxxx" <xuzaibo@xxxxxxxxxx>, > > "linux-accelerators@xxxxxxxxxxxxxxxx" > > <linux-accelerators@xxxxxxxxxxxxxxxx> > > Subject: Re: To extend the feature of vfio-mdev > > User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 > > Thunderbird/52.4.0 > > Message-ID: <6fed6c27-4b38-7d65-ee5d-63d6bb283e95@xxxxxxx> > > > > On 23/10/17 06:18, Kenneth Lee wrote: > > >>>>> 3. Let iommu driver accept more than one iommu_domain for the same > > >>>>> device. The substream id or pasid should be support for that > > >>>> > > >>>> You're really extending the definition of an iommu_domain to include > > >>>> PASID to do this, I don't think it makes sense in the general case. So > > >>>> perhaps you're talking about a PASID management layer sitting on top of > > >>>> an iommu_domain. AIUI for PCIe, a device has a requester ID which is > > >>>> used to find the context entry for that device. The IOMMU may support > > >>>> PASID, which would cause a first level lookup via those set of page > > >>>> tables, or it might only support second level translation. The > > >>>> iommu_domain is a reflection of that initial, single requester ID. > > >>> > > >>> Maybe I misunderstand this. But the IOMMU hardware, such as SMMU for ARM, > > >>> support multiple page table and is referred by something like ASID. If we should > > >>> support it in Linux, iommu_domain should be the best choice (no matter you call > > >>> it cookie or id or something else). Or where you can get a object referring to it? > > >> > > >> For PASID, a PASID is unique only within the requester ID. I don't > > >> know of anything equivalent to your ASID within PCIe. > > >> > > > > > > As my understanding to the ARM Spec: > > > > > > PCIE[PASID]=ARM_SMMU[ASID]=ARM_SMMU[SubstreamID], > > > PCIE[RequestID]=ARM_SMMU[StreamID] > > > > In the SMMUv3 spec, the PASID is dissociated from the ASID. We set the > > ASID in the context descriptor indexed by PASID, which provides more > > flexibility when the CPU's ASID capacity and endpoint's PASID capacity differ. > > Yes. Sorry, I tried to make the concept simply. But it is wrong slightly. > > > > > > For ARM SMMU, the Stream ID is used to index a Context Descriptors > > > Table while and Sub-stream ID is used to index a Descriptor which refer > > > to a general page table in the same format as MMU. > > > > > > So both RequestID and PASID uniquely identify a address space. Then > > > every iommu-enabled device can service more than one user process at the > > > same time. > > > > > > So the IOMMU should support more than one page table, which in turn > > > should be added to somewhere in Linux. If the iommu_domain refer to one > > > address space, the iommu driver should accept more than one > > > iommu_domain. > > > > The current design choice is to have multiple address space per > > iommu_domain (and one PASID table per domain). It fits better with > > existing IOMMU and VFIO APIs. > > > > I am still reading the patchset. But if the PASID is used to index the address > space table in iommu_domain, it will become a general concept in the whole > system. But if we would bind the address space with iommu_domain, the > iommu_domain become a handle for any process to control the device dma. It will > be easier for one process with two or more domains to control the device for > different purpose. And it is also easier for a process hand over the domain to > another without change anything. And the process need not to know anything about > pasid. A domain is a handle to control the device separately. I think this make > thing simpler. > > > Thanks, > > Jean > > Thanks for discussion. > -- > -Kenneth(Hisilicon) Hi, Jean, I read your patchset on git://linux-arm.org/linux-jpb in branch svm/rfc2. I don't understand why you bind the multiple asid/pasid feature with SVM. I think it is risky. The performance of SVM is still a risk to many hardware. It rely on the workflow. In many cases, we just need to create a page table particularly for the devices itself without dedicate the whole process space to it. So why not just make them as two features? binding a new address space to the iommu_process or dedicate the whole process address space to it. Best Regards -- -Kenneth Lee (Hisilicon)
