Hi Shameer, On 6/28/22 10:14, Shameerali Kolothum Thodi wrote: > >> -----Original Message----- >> From: Yi Liu [mailto:yi.l.liu@xxxxxxxxx] >> Sent: 18 May 2022 15:01 >> To: zhangfei.gao@xxxxxxxxxxx; Jason Gunthorpe <jgg@xxxxxxxxxx>; >> Zhangfei Gao <zhangfei.gao@xxxxxxxxxx> >> Cc: eric.auger@xxxxxxxxxx; Alex Williamson <alex.williamson@xxxxxxxxxx>; >> Shameerali Kolothum Thodi <shameerali.kolothum.thodi@xxxxxxxxxx>; >> cohuck@xxxxxxxxxx; qemu-devel@xxxxxxxxxx; >> david@xxxxxxxxxxxxxxxxxxxxx; thuth@xxxxxxxxxx; farman@xxxxxxxxxxxxx; >> mjrosato@xxxxxxxxxxxxx; akrowiak@xxxxxxxxxxxxx; pasic@xxxxxxxxxxxxx; >> jjherne@xxxxxxxxxxxxx; jasowang@xxxxxxxxxx; kvm@xxxxxxxxxxxxxxx; >> nicolinc@xxxxxxxxxx; eric.auger.pro@xxxxxxxxx; kevin.tian@xxxxxxxxx; >> chao.p.peng@xxxxxxxxx; yi.y.sun@xxxxxxxxx; peterx@xxxxxxxxxx >> Subject: Re: [RFC 00/18] vfio: Adopt iommufd >> >> On 2022/5/18 15:22, zhangfei.gao@xxxxxxxxxxx wrote: >>> >>> On 2022/5/17 下午4:55, Yi Liu wrote: >>>> Hi Zhangfei, >>>> >>>> On 2022/5/12 17:01, zhangfei.gao@xxxxxxxxxxx wrote: >>>>> Hi, Yi >>>>> >>>>> On 2022/5/11 下午10:17, zhangfei.gao@xxxxxxxxxxx wrote: >>>>>> >>>>>> On 2022/5/10 下午10:08, Yi Liu wrote: >>>>>>> On 2022/5/10 20:45, Jason Gunthorpe wrote: >>>>>>>> On Tue, May 10, 2022 at 08:35:00PM +0800, Zhangfei Gao wrote: >>>>>>>>> Thanks Yi and Eric, >>>>>>>>> Then will wait for the updated iommufd kernel for the PCI MMIO >> region. >>>>>>>>> Another question, >>>>>>>>> How to get the iommu_domain in the ioctl. >>>>>>>> The ID of the iommu_domain (called the hwpt) it should be returned >> by >>>>>>>> the vfio attach ioctl. >>>>>>> yes, hwpt_id is returned by the vfio attach ioctl and recorded in >>>>>>> qemu. You can query page table related capabilities with this id. >>>>>>> >>>>>>> >> https://lore.kernel.org/kvm/20220414104710.28534-16-yi.l.liu@xxxxxxxxx/ >>>>>> Thanks Yi, >>>>>> >>>>>> Do we use iommufd_hw_pagetable_from_id in kernel? >>>>>> >>>>>> The qemu send hwpt_id via ioctl. >>>>>> Currently VFIOIOMMUFDContainer has hwpt_list, >>>>>> Which member is good to save hwpt_id, IOMMUTLBEntry? >>>>> Can VFIOIOMMUFDContainer have multi hwpt? >>>> yes, it is possible >>> Then how to get hwpt_id in map/unmap_notify(IOMMUNotifier *n, >> IOMMUTLBEntry >>> *iotlb) >> in map/unmap, should use ioas_id instead of hwpt_id >> >>>>> Since VFIOIOMMUFDContainer has hwpt_list now. >>>>> If so, how to get specific hwpt from map/unmap_notify in hw/vfio/as.c, >>>>> where no vbasedev can be used for compare. >>>>> >>>>> I am testing with a workaround, adding VFIOIOASHwpt *hwpt in >>>>> VFIOIOMMUFDContainer. >>>>> And save hwpt when vfio_device_attach_container. >>>>> >>>>>> In kernel ioctl: iommufd_vfio_ioctl >>>>>> @dev: Device to get an iommu_domain for >>>>>> iommufd_hw_pagetable_from_id(struct iommufd_ctx *ictx, u32 pt_id, >>>>>> struct device *dev) >>>>>> But iommufd_vfio_ioctl seems no para dev? >>>>> We can set dev=Null since IOMMUFD_OBJ_HW_PAGETABLE does not >> need dev. >>>>> iommufd_hw_pagetable_from_id(ictx, hwpt_id, NULL) >>>> this is not good. dev is passed in to this function to allocate domain >>>> and also check sw_msi things. If you pass in a NULL, it may even unable >>>> to get a domain for the hwpt. It won't work I guess. >>> The iommufd_hw_pagetable_from_id can be used for >>> 1, allocate domain, which need para dev >>> case IOMMUFD_OBJ_IOAS >>> hwpt = iommufd_hw_pagetable_auto_get(ictx, ioas, dev); >> this is used when attaching ioas. >> >>> 2. Just return allocated domain via hwpt_id, which does not need dev. >>> case IOMMUFD_OBJ_HW_PAGETABLE: >>> return container_of(obj, struct iommufd_hw_pagetable, obj); >> yes, this would be the usage in nesting. you may check my below >> branch. It's for nesting integration. >> >> https://github.com/luxis1999/iommufd/tree/iommufd-v5.18-rc4-nesting >> >>> By the way, any plan of the nested mode? >> I'm working with Eric, Nic on it. Currently, I've got the above kernel >> branch, QEMU side is also WIP. > Hi Yi/Eric, > > I had a look at the above nesting kernel and Qemu branches and as mentioned > in the cover letter it is not working on ARM yet. > > IIUC, to get it working via the iommufd the main thing is we need a way to configure > the phys SMMU in nested mode and setup the mappings for the stage 2. The > Cache/PASID related changes looks more straight forward. > > I had quite a few hacks to get it working on ARM, but still a WIP. So just wondering > do you guys have something that can be shared yet? I am working on the respin based on latest iommufd kernel branches and qemu RFC v2 but it is still WIP. I will share as soon as possible. Eric > > Please let me know. > > Thanks, > Shameer
