On Wed, Jul 20, 2016 at 01:03:00PM +0200, Auger Eric wrote: > Hi Dennis > On 20/07/2016 11:56, Dennis Chen wrote: > > Hi Eric, > > > > On Tue, Jul 19, 2016 at 12:55:03PM +0000, Eric Auger wrote: > >> This series introduces the msi-iommu api used to: > >> > >> - allocate/free resources for MSI IOMMU mapping > >> - set the MSI iova window aperture > >> - map/unmap physical addresses onto MSI IOVAs. > >> - determine whether an msi needs to be iommu mapped > >> - overwrite an msi_msg PA address with its pre-allocated/mapped IOVA > >> > >> Also a new iommu domain attribute, DOMAIN_ATTR_MSI_GEOMETRY is introduced > >> to report the MSI iova window geometry (aperture and iommu-msi API support). > >> > >> Currently: > >> - iommu driver is supposed to allocate/free MSI mapping resources > >> - VFIO subsystem is supposed to set the MSI IOVA aperture. > >> - The MSI layer is supposed to allocate/free iova mappings and overwrite > >> msi_msg with IOVA at composition time > >> > >> More details & context can be found at: > >> http://www.linaro.org/blog/core-dump/kvm-pciemsi-passthrough-armarm64/ > >> > >> Best Regards > >> > >> Eric > >> > >> Git: complete series available at > >> https://github.com/eauger/linux/tree/v4.7-rc7-passthrough-v11 > >> > > Why can't I find this new series on your git tree: > > https://git.linaro.org/people/eric.auger/linux.git > you are not looking at the right git repo: see github one above. > > ? > > Also, do I need to download all the 3-part patches to test the PCIe NIC passthru > > as I did on your v9 series? > Yes you need to take the 3 parts. You should have everything that is > needed on the above branch. In case you do not work on Cavium, you > should not cherry-pick > "vfio: pci: HACK! workaround thunderx pci_try_reset_bus crash" > Thanks Eric, I've got all the 3 parts from the github git tree in my local repos. Currently I think the major platform of mine is AMD overdrive. So I will test this patch set on that platform first. Thanks, Dennis > > Thanks > > Eric > > > > Thanks, > > Dennis > >> > >> see part III for wrap-up details. > >> > >> History: > >> v10 -> v11: > >> - no change in the series, just incremented for consistency > >> - added a temporary patch in the branch: > >> "iommu/iova: FIXUP! validate iova_domain input to put_iova_domain" > >> originally sent by Nate and adapted for this use case. This is currently > >> under discussion on the ML. The crash typically occurs in case unsafe > >> interrupts are discovered while allow_unsafe_interrupts is not set. > >> > >> v9 -> v10: > >> - split error management in iommu_msi_set_aperture > >> > >> v8 -> v9: > >> - rename iommu_domain_msi_geometry programmable flag into iommu_msi_supported > >> - introduce msi_apperture_valid helper and use this instead of is_aperture_set > >> > >> v7 -> v8: > >> - The API is retargetted for MSI: renamed msi-iommu > >> all "dma-reserved" namings removed > >> - now implemented upon dma-iommu (get, put, init), ie. reuse iova_cookie, > >> and iova API > >> - msi mapping resources now are guaranteed to exist during the whole iommu > >> domain's lifetime. No need to lock to garantee the cookie integrity > >> - removed alloc/free_reserved_reserved_iova_domain. We now have a single > >> function that sets the aperture, looking like iommu_dma_init_domain. > >> - we now use a list instead of an RB-tree > >> - prot is not propagated anymore at domain creation due to the retargetting > >> for MSI > >> - iommu_domain pointer removed from doorbell_mapping struct > >> - replaced DOMAIN_ATTR_MSI_MAPPING by DOMAIN_ATTR_MSI_GEOMETRY > >> > >> v6 -> v7: > >> - fixed known lock bugs and multiple page sized slots matching > >> (I only have a single MSI frame made of a single page) > >> - reserved_iova_cookie now pointing to a struct that encapsulates the > >> iova domain handle + protection attribute passed from VFIO (Alex' req) > >> - 2 new functions exposed: iommu_msi_mapping_translate_msg, > >> iommu_msi_mapping_desc_to_domain: not sure this is the right location/proto > >> though > >> - iommu_put_reserved_iova now takes a phys_addr_t > >> - everything now is cleanup on iommu_domain destruction > >> > >> RFC v5 -> patch v6: > >> - split to ease the review process > >> - in dma-reserved-api use a spin lock instead of a mutex (reported by > >> Jean-Philippe) > >> - revisit iommu_get_reserved_iova API to pass a size parameter upon > >> Marc's request > >> - Consistently use the page order passed when creating the iova domain. > >> - init reserved_binding_list (reported by Julien) > >> > >> RFC v4 -> RFC v5: > >> - take into account Thomas' comments on MSI related patches > >> - split "msi: IOMMU map the doorbell address when needed" > >> - increase readability and add comments > >> - fix style issues > >> - split "iommu: Add DOMAIN_ATTR_MSI_MAPPING attribute" > >> - platform ITS now advertises IOMMU_CAP_INTR_REMAP > >> - fix compilation issue with CONFIG_IOMMU API unset > >> - arm-smmu-v3 now advertises DOMAIN_ATTR_MSI_MAPPING > >> > >> RFC v3 -> v4: > >> - Move doorbell mapping/unmapping in msi.c > >> - fix ref count issue on set_affinity: in case of a change in the address > >> the previous address is decremented > >> - doorbell map/unmap now is done on msi composition. Should allow the use > >> case for platform MSI controllers > >> - create dma-reserved-iommu.h/c exposing/implementing a new API dedicated > >> to reserved IOVA management (looking like dma-iommu glue) > >> - series reordering to ease the review: > >> - first part is related to IOMMU > >> - second related to MSI sub-system > >> - third related to VFIO (except arm-smmu IOMMU_CAP_INTR_REMAP removal) > >> - expose the number of requested IOVA pages through VFIO_IOMMU_GET_INFO > >> [this partially addresses Marc's comments on iommu_get/put_single_reserved > >> size/alignment problematic - which I did not ignore - but I don't know > >> how much I can do at the moment] > >> > >> RFC v2 -> RFC v3: > >> - should fix wrong handling of some CONFIG combinations: > >> CONFIG_IOVA, CONFIG_IOMMU_API, CONFIG_PCI_MSI_IRQ_DOMAIN > >> - fix MSI_FLAG_IRQ_REMAPPING setting in GICv3 ITS (although not tested) > >> > >> PATCH v1 -> RFC v2: > >> - reverted to RFC since it looks more reasonable ;-) the code is split > >> between VFIO, IOMMU, MSI controller and I am not sure I did the right > >> choices. Also API need to be further discussed. > >> - iova API usage in arm-smmu.c. > >> - MSI controller natively programs the MSI addr with either the PA or IOVA. > >> This is not done anymore in vfio-pci driver as suggested by Alex. > >> - check irq remapping capability of the group > >> > >> RFC v1 [2] -> PATCH v1: > >> - use the existing dma map/unmap ioctl interface with a flag to register a > >> reserved IOVA range. Use the legacy Rb to store this special vfio_dma. > >> - a single reserved IOVA contiguous region now is allowed > >> - use of an RB tree indexed by PA to store allocated reserved slots > >> - use of a vfio_domain iova_domain to manage iova allocation within the > >> window provided by the userspace > >> - vfio alloc_map/unmap_free take a vfio_group handle > >> - vfio_group handle is cached in vfio_pci_device > >> - add ref counting to bindings > >> - user modality enabled at the end of the series > >> > >> > >> Eric Auger (8): > >> iommu: Add iommu_domain_msi_geometry and DOMAIN_ATTR_MSI_GEOMETRY > >> iommu/arm-smmu: initialize the msi geometry and advertise iommu-msi > >> support > >> iommu: introduce an msi cookie > >> iommu/msi-iommu: initialization > >> iommu/msi-iommu: iommu_msi_[get,put]_doorbell_iova > >> iommu/msi-iommu: iommu_msi_domain > >> iommu/msi-iommu: iommu_msi_msg_pa_to_va > >> iommu/arm-smmu: get/put the msi cookie > >> > >> drivers/iommu/Kconfig | 7 + > >> drivers/iommu/Makefile | 1 + > >> drivers/iommu/arm-smmu-v3.c | 18 ++- > >> drivers/iommu/arm-smmu.c | 18 ++- > >> drivers/iommu/iommu.c | 5 + > >> drivers/iommu/msi-iommu.c | 322 ++++++++++++++++++++++++++++++++++++++++++++ > >> include/linux/iommu.h | 15 +++ > >> include/linux/msi-iommu.h | 144 ++++++++++++++++++++ > >> 8 files changed, 522 insertions(+), 8 deletions(-) > >> create mode 100644 drivers/iommu/msi-iommu.c > >> create mode 100644 include/linux/msi-iommu.h > >> > >> -- > >> 1.9.1 > >> > >> _______________________________________________ > >> kvmarm mailing list > >> kvmarm@xxxxxxxxxxxxxxxxxxxxx > >> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm > >> > > > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
