I've been struggling with a problem for a while and I haven't been able to come up with a clean solution. Rob convinced me to stop complaining and do _something_ and hopefully this can spur a good discussion. The scenario is basically this: The MSM GPU wants to manage its own IOMMU virtual address space in lieu of using the DMA API to do so. The most important reason is that when per-instance pagetables [1] are enabled each file descriptor uses its own pagetable which are dynamically switched between contexts. In conjunction with this we use a split pagetable scheme to allow global buffers be persistently mapped so even a single pagetable has multiple ranges that need to be utilized based on buffer type. Obviously the DMA API isn't suitable for this kind of specialized use. We want to push it off to the side, allocate our own domain, and use it directly with the IOMMU APIs. In a perfect world we would just not use the DMA API and attach our own domain and that would be the end of the story. Unfortunately this is not a perfect world. In order to switch the pagetable from the GPU the SoC has some special wiring so that the GPU can reprogram the TTBR0 register in the IOMMU to the appropriate value. For a variety of reasons the hardware is hardcoded to only be able to access context bank 0 in the SMMU device. Long story short; if the DMA domain is allocated during bus enumeration and attached to context bank 0 then by the time we get to the driver domain we have long since lost our chance to get the context bank we need. After going back and forth and trying a few possible options it seems like the "easiest" solution" is to skip attaching the DMA domain in the first place. But we still need to create a default domain and set up the IOMMU groups correctly so that when the GPU driver comes along it can attach the device and everything will Just Work (TM). Rob suggested that we should use a blacklist of devices that choose to not participate so thats what I'm offering as a starting point for discussion. The first patch in this series allows the specific IOMMU driver to gracefully skip attaching a device by returning -ENOSUPP. In that case, the core will skip printing error messages and it won't attach the domain to the group but it still allows the group to get created so that the IOMMU device is properly set up. In arch_setup_dma_ops() the call to iommu_get_domain_for_dev() will return NULL and the IOMMU ops won't be set up. The second patch implements the blacklist in arm-smmu.c based on the compatible string of the GPU device and returns -ENOTSUPP. I tested this with my current per-instance pagetable stack and it does the job but I am very much in the market for better ideas. [1] https://patchwork.freedesktop.org/series/38729/ Jordan Crouse (2): iommu: Gracefully allow drivers to not attach to a default domain iommu/arm-smmu: Add list of devices to opt out of DMA domains drivers/iommu/arm-smmu.c | 23 +++++++++++++++++++++++ drivers/iommu/iommu.c | 18 ++++++++++++++---- 2 files changed, 37 insertions(+), 4 deletions(-) -- 2.16.1 -- To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
