On Tue, 8 May 2018 14:49:23 -0600 Logan Gunthorpe <logang@xxxxxxxxxxxx> wrote: > On 08/05/18 02:43 PM, Alex Williamson wrote: > > Yes, GPUs seem to be leading the pack in implementing ATS. So now the > > dumb question, why not simply turn off the IOMMU and thus ACS? The > > argument of using the IOMMU for security is rather diminished if we're > > specifically enabling devices to poke one another directly and clearly > > this isn't favorable for device assignment either. Are there target > > systems where this is not a simple kernel commandline option? Thanks, > > Well, turning off the IOMMU doesn't necessarily turn off ACS. We've run > into some bios's that set the bits on boot (which is annoying). But it would be a much easier proposal to disable ACS when the IOMMU is not enabled, ACS has no real purpose in that case. > I also don't expect people will respond well to making the IOMMU and P2P > exclusive. The IOMMU is often used for more than just security and on > many platforms it's enabled by default. I'd much rather allow IOMMU use > but have fewer isolation groups in much the same way as if you had PCI > bridges that didn't support ACS. The IOMMU and P2P are already not exclusive, we can bounce off the IOMMU or make use of ATS as we've previously discussed. We were previously talking about a build time config option that you didn't expect distros to use, so I don't think intervention for the user to disable the IOMMU if it's enabled by default is a serious concern either. What you're trying to do is enabled direct peer-to-peer for endpoints which do not support ATS when the IOMMU is enabled, which is not something that necessarily makes sense to me. As I mentioned in a previous reply, the IOMMU provides us with an I/O virtual address space for devices, ACS is meant to fill the topology based gaps in that virtual address space, making transactions follow IOMMU compliant routing rules to avoid aliases between the IOVA and physical address spaces. But this series specifically wants to leave those gaps open for direct P2P access. So we compromise the P2P aspect of security, still protecting RAM, but potentially only to the extent that a device cannot hop through or interfere with other devices to do its bidding. Device assignment is mostly tossed out the window because not only are bigger groups more difficult to deal with, the IOVA space is riddled with gaps, which is not really a solved problem. So that leaves avoiding bounce buffers as the remaining IOMMU feature, but we're dealing with native express devices and relatively high end devices that are probably installed in modern systems, so that seems like a non-issue. Are there other uses I'm forgetting? We can enable interrupt remapping separate from DMA translation, so we can exclude that one. I'm still not seeing why it's terribly undesirable to require devices to support ATS if they want to do direct P2P with an IOMMU enabled. Thanks, Alex
