On Mon, May 15, 2023 at 6:23 PM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > On Mon, May 08, 2023 at 07:58:38PM +0530, Anup Patel wrote: > > We have a separate RISC-V IMSIC MSI address for each CPU so changing > > MSI (or IRQ) affinity results in re-programming of MSI address in > > the PCIe (or platform) device. > > > > Currently, the iommu_dma_prepare_msi() is called only once at the > > time of IRQ allocation so IOMMU DMA domain will only have mapping > > for one MSI page. This means iommu_dma_compose_msi_msg() called > > by imsic_irq_compose_msi_msg() will always use the same MSI page > > irrespective to target CPU MSI address. In other words, changing > > MSI (or IRQ) affinity for device using IOMMU DMA domain will not > > work. > > > > To address above issue, we do the following: > > 1) Map MSI pages for all CPUs in imsic_irq_domain_alloc() > > using iommu_dma_prepare_msi(). > > 2) Add a new iommu_dma_select_msi() API to select a specific > > MSI page from a set of already mapped MSI pages. > > 3) Use iommu_dma_select_msi() to select a specific MSI page > > before calling iommu_dma_compose_msi_msg() in > > imsic_irq_compose_msi_msg(). > > Is there an iommu driver somewhere in all this? I don't obviously see > one? Sorry for the delayed response. The RISC-V IOMMU specification is frozen and will be ratified/released anytime this month or next. https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0-rc6/riscv-iommu.pdf The RISC-V IOMMU driver will be send-out on LKML pretty soon https://github.com/tjeznach/linux/tree/tjeznach/riscv-iommu which can be tested on QEMU https://github.com/tjeznach/qemu/tree/tjeznach/riscv-iommu > > There should be no reason to use the dma-iommu.c stuff just to make > interrupts work, that is only necessary if there is an iommu, and the > platform architecture requires the iommu to have the MSI region > programmed into IOPTEs. > > And I'd be much happier if we could clean this design up before risc-v > starts using it too :\ > Sure, I will send-out v4 in the next few days. Regards, Anup
