Hi Alex, On 04/07/2016 01:15 AM, Alex Williamson wrote: > On Mon, 4 Apr 2016 08:06:55 +0000 > Eric Auger <eric.auger@xxxxxxxxxx> wrote: > >> This series introduces the dma-reserved-iommu api used to: >> - create/destroy an iova domain dedicated to reserved iova bindings >> - map/unmap physical addresses onto reserved IOVAs. >> - unmap and destroy all IOVA reserved bindings > > Why are we making the decision to have an unbalanced map vs unmap, we > can create individual mappings, but only unmap the whole thing and > start over? That's a strange interface. Thanks, The "individual" balanced unmap also exists (iommu_put_reserved_iova) and this is the "normal" path. This happens on msi_domain_deactivate (and possibly on msi_domain_set_affinity). I added iommu_unmap_reserved to handle the case where the userspace registers a reserved iova domain and fails to unregister it. In that case one need to handle the cleanup on kernel-side and I chose to implement this on vfio_iommu_type1 release. All the reserved IOMMU bindings get destroyed on that event. Any advice to handle this situation? Best Regards Eric > > Alex > >> Currently reserved IOVAs are meant to map MSI physical doorbells. A single >> reserved domain does exit per domain. >> >> Also a new domain attribute is introduced to signal whether the MSI >> addresses must be mapped in the IOMMU >> >> VFIO subsystem is supposed to create/destroy the iommu reserved domain. >> >> When the MSI sub-system is about to handle an MSI physical address >> that needs to be bound, it uses the dma-reserved_iommu API to map/unmap >> the address. Since several drivers are likely to use the same doorbell, >> a reference counting must exist on the bindings. An RB-tree indexed by PA >> is used. >> >> More details & context can be found at: >> http://www.linaro.org/blog/core-dump/kvm-pciemsi-passthrough-armarm64/ >> >> Best Regards >> >> Eric >> >> Git: complete series available at >> https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.6-rc1-pcie-passthrough-v6 >> >> History: >> >> RFC v5 -> patch v6: >> - split to ease the review process >> - in dma-reserved-api use a spin lock instead of a mutex (reported by >> Jean-Philippe) >> - revisit iommu_get_reserved_iova API to pass a size parameter upon >> Marc's request >> - Consistently use the page order passed when creating the iova domain. >> - init reserved_binding_list (reported by Julien) >> >> RFC v4 -> RFC v5: >> - take into account Thomas' comments on MSI related patches >> - split "msi: IOMMU map the doorbell address when needed" >> - increase readability and add comments >> - fix style issues >> - split "iommu: Add DOMAIN_ATTR_MSI_MAPPING attribute" >> - platform ITS now advertises IOMMU_CAP_INTR_REMAP >> - fix compilation issue with CONFIG_IOMMU API unset >> - arm-smmu-v3 now advertises DOMAIN_ATTR_MSI_MAPPING >> >> RFC v3 -> v4: >> - Move doorbell mapping/unmapping in msi.c >> - fix ref count issue on set_affinity: in case of a change in the address >> the previous address is decremented >> - doorbell map/unmap now is done on msi composition. Should allow the use >> case for platform MSI controllers >> - create dma-reserved-iommu.h/c exposing/implementing a new API dedicated >> to reserved IOVA management (looking like dma-iommu glue) >> - series reordering to ease the review: >> - first part is related to IOMMU >> - second related to MSI sub-system >> - third related to VFIO (except arm-smmu IOMMU_CAP_INTR_REMAP removal) >> - expose the number of requested IOVA pages through VFIO_IOMMU_GET_INFO >> [this partially addresses Marc's comments on iommu_get/put_single_reserved >> size/alignment problematic - which I did not ignore - but I don't know >> how much I can do at the moment] >> >> RFC v2 -> RFC v3: >> - should fix wrong handling of some CONFIG combinations: >> CONFIG_IOVA, CONFIG_IOMMU_API, CONFIG_PCI_MSI_IRQ_DOMAIN >> - fix MSI_FLAG_IRQ_REMAPPING setting in GICv3 ITS (although not tested) >> >> PATCH v1 -> RFC v2: >> - reverted to RFC since it looks more reasonable ;-) the code is split >> between VFIO, IOMMU, MSI controller and I am not sure I did the right >> choices. Also API need to be further discussed. >> - iova API usage in arm-smmu.c. >> - MSI controller natively programs the MSI addr with either the PA or IOVA. >> This is not done anymore in vfio-pci driver as suggested by Alex. >> - check irq remapping capability of the group >> >> RFC v1 [2] -> PATCH v1: >> - use the existing dma map/unmap ioctl interface with a flag to register a >> reserved IOVA range. Use the legacy Rb to store this special vfio_dma. >> - a single reserved IOVA contiguous region now is allowed >> - use of an RB tree indexed by PA to store allocated reserved slots >> - use of a vfio_domain iova_domain to manage iova allocation within the >> window provided by the userspace >> - vfio alloc_map/unmap_free take a vfio_group handle >> - vfio_group handle is cached in vfio_pci_device >> - add ref counting to bindings >> - user modality enabled at the end of the series >> >> >> Eric Auger (7): >> iommu: Add DOMAIN_ATTR_MSI_MAPPING attribute >> iommu/arm-smmu: advertise DOMAIN_ATTR_MSI_MAPPING attribute >> iommu: introduce a reserved iova cookie >> dma-reserved-iommu: alloc/free_reserved_iova_domain >> dma-reserved-iommu: reserved binding rb-tree and helpers >> dma-reserved-iommu: iommu_get/put_single_reserved >> dma-reserved-iommu: iommu_unmap_reserved >> >> drivers/iommu/Kconfig | 8 + >> drivers/iommu/Makefile | 1 + >> drivers/iommu/arm-smmu-v3.c | 2 + >> drivers/iommu/arm-smmu.c | 2 + >> drivers/iommu/dma-reserved-iommu.c | 321 +++++++++++++++++++++++++++++++++++++ >> drivers/iommu/iommu.c | 2 + >> include/linux/dma-reserved-iommu.h | 80 +++++++++ >> include/linux/iommu.h | 7 + >> 8 files changed, 423 insertions(+) >> create mode 100644 drivers/iommu/dma-reserved-iommu.c >> create mode 100644 include/linux/dma-reserved-iommu.h >> > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
