在 2021/6/3 上午4:37, Alex Williamson 写道:
On Wed, 2 Jun 2021 16:54:04 -0300
Jason Gunthorpe <jgg@xxxxxxxxxx> wrote:
On Wed, Jun 02, 2021 at 01:00:53PM -0600, Alex Williamson wrote:
Right, the device can generate the no-snoop transactions, but it's the
IOMMU that essentially determines whether those transactions are
actually still cache coherent, AIUI.
Wow, this is really confusing stuff in the code.
At the PCI level there is a TLP bit called no-snoop that is platform
specific. The general intention is to allow devices to selectively
bypass the CPU caching for DMAs. GPUs like to use this feature for
performance.
Yes
I assume there is some exciting security issues here. Looks like
allowing cache bypass does something bad inside VMs? Looks like
allowing the VM to use the cache clear instruction that is mandatory
with cache bypass DMA causes some QOS issues? OK.
IIRC, largely a DoS issue if userspace gets to choose when to emulate
wbinvd rather than it being demanded for correct operation.
So how does it work?
What I see in the intel/iommu.c is that some domains support "snoop
control" or not, based on some HW flag. This indicates if the
DMA_PTE_SNP bit is supported on a page by page basis or not.
Since x86 always leans toward "DMA cache coherent" I'm reading some
tea leaves here:
IOMMU_CAP_CACHE_COHERENCY, /* IOMMU can enforce cache coherent DMA
transactions */
And guessing that IOMMUs that implement DMA_PTE_SNP will ignore the
snoop bit in TLPs for IOVA's that have DMA_PTE_SNP set?
That's my understanding as well.
Further, I guess IOMMUs that don't support PTE_SNP, or have
DMA_PTE_SNP clear will always honour the snoop bit. (backwards compat
and all)
Yes.
So, IOMMU_CAP_CACHE_COHERENCY does not mean the IOMMU is DMA
incoherent with the CPU caches, it just means that that snoop bit in
the TLP cannot be enforced. ie the device *could* do no-shoop DMA
if it wants. Devices that never do no-snoop remain DMA coherent on
x86, as they always have been.
Yes, IOMMU_CAP_CACHE_COHERENCY=false means we cannot force the device
DMA to be coherent via the IOMMU.
IOMMU_CACHE does not mean the IOMMU is DMA cache coherent, it means
the PCI device is blocked from using no-snoop in its TLPs.
I wonder if ARM implemented this consistently? I see VDPA is
confused..
Basically, we don't want to bother with pseudo KVM device like what VFIO
did. So for simplicity, we rules out the IOMMU that can't enforce
coherency in vhost-vDPA if the parent purely depends on the platform IOMMU:
if (!iommu_capable(bus, IOMMU_CAP_CACHE_COHERENCY))
return -ENOTSUPP;
For the parents that use its own translations logic, an implicit
assumption is that the hardware will always perform cache coherent DMA.
Thanks
