On Thu, Oct 21, 2021 at 08:22:23PM -0300, Jason Gunthorpe wrote: > On Thu, Oct 21, 2021 at 03:58:02PM +0100, Jean-Philippe Brucker wrote: > > On Thu, Oct 21, 2021 at 02:26:00AM +0000, Tian, Kevin wrote: > > > > I'll leave it to Jean to confirm. If only coherent DMA can be used in > > > > the guest on other platforms, suppose VFIO should not blindly set > > > > IOMMU_CACHE and in concept it should deny assigning a non-coherent > > > > device since no co-ordination with guest exists today. > > > > > > Jean, what's your opinion? > > > > Yes a sanity check to prevent assigning non-coherent devices would be > > good, though I'm not particularly worried about non-coherent devices. PCIe > > on Arm should be coherent (according to the Base System Architecture). So > > vfio-pci devices should be coherent, but vfio-platform and mdev are > > case-by-case (hopefully all coherent since it concerns newer platforms). > > > > More worrying, I thought we disabled No-Snoop for VFIO but I was wrong, > > it's left enabled. On Arm I don't think userspace can perform the right > > cache maintenance operations to maintain coherency with a device that > > issues No-Snoop writes. Userspace can issue clean+invalidate but not > > invalidate alone, so there is no equivalent to > > arch_sync_dma_for_cpu(). > > So what happens in a VM? Does a VM know that arch_sync_dma_for_cpu() > is not available? This would only affect userspace drivers, it's only host or guest userspace that cannot issue the maintenance operations. The VM can do arch_sync_dma_for_cpu() Thanks, Jean > > And how does this work with the nested IOMMU translation? I thought I > read in the SMMU spec that the io page table entries could control > cachability including in nesting cases? > > > I think the worse that can happen is the device owner shooting itself in > > the foot by using No-Snoop, but would it hurt to disable it? > > No, the worst is the same as Intel - a driver running in the guest VM > assumes it can use arch_sync_dma_for_cpu() and acts accordingly, > resulting in a broken VM. > > Jason
