On Thu, May 9, 2019 at 1:03 PM Gary R Hook <ghook@xxxxxxx> wrote: > > On 4/24/19 5:04 PM, Eric Pilmore wrote: > > > > Thanks the for the response. We are using the correct device for the > > dma_alloc_coherent(). Upon further investigation what we are finding > > is that apparently the AMD IOMMU support can only manage one alias, as > > opposed to Intel IOMMU support which can support multiple. Not clear > > at this time if it's a software limitation in the AMD IOMMU kernel > > support or an imposed limitation of the hardware. Still investigating. > > Please define 'alias'? Hi Gary, I appreciate the response. Sorry for the late reply. Got sidetracked with other stuff. I will try to answer this as best I can. Sorry if my terminology might be off as I'm still a relative newbie with some of this. The "alias" is basically another BDF (or ProxyID) that wants to be associated with the same IOMMU resources as some primary BDF. Reference <drivers/pci/quirks.c>. In the scenario that we have we are utilizing NTB and through this bridge will come requests (TLPs) that will not necessarily have the ReqID as the BDF of the switch device that contains this bridge. Instead, the ReqID will be a "translated" (Proxy) BDF of sourcing devices on the other side of the Non-Transparent Bridge. In our case our NTB is a Switchtec device and the quirk quirk_switchtec_ntb_dma_alias() is used as a means of associating these aliases (aka ProxyID or Translated ReqID) with the NT endpoint in the local host. On Xeon platforms, the framework supports allowing multiple aliases to be defined for a particular IOMMU and everything works great. However, with the AMD cpu, it appears the IOMMU framework is only accepting just one alias. Note Logan's earlier response @ Mon, Apr 22, 10:31 AM. In our case the one that is accepted is via the path for a processor Read, but Processor Writes go through a slightly different path resulting in a different ReqID. As Logan points out it seems since the AMD IOMMU code is only accepting one alias, the Write ReqID looks foreign and thus results in the IOMMU faults. > > The IO_PAGE_FAULT error is described on page 142 of the AMD IOMMU spec, > document #48882. Easily found via a search. > > The flags value of 0x0070 translates to PE, RW, PR. The page was > present, the transaction was a write, and the peripheral didn't have > permission. That implies that mapping hadn't been done. > > Not being sure how that device presents, or what you're doing with IVHD > info, I can't comment further. I can say that the AMD IOMMU provides for > a single exclusion range, but as many unity ranges as you wish. I'm currently not doing anything with IVHD. The devices on the other side of the NTB that need to be aliased can be anything from a remote Host processor, NVMe drive, GPU, etc., anything that wants to send a memory transaction to the local host. If you have any insight into how the AMD IOMMU support in the kernel could be extended for multiple aliases, or whether there is a hardware limitation that restricts it to just one, that would be greatly appreciated. Thanks, Eric -- Eric Pilmore epilmore@xxxxxxxxxx http://gigaio.com Phone: (858) 775 2514 This e-mail message is intended only for the individual(s) to whom it is addressed and may contain information that is privileged, confidential, proprietary, or otherwise exempt from disclosure under applicable law. If you believe you have received this message in error, please advise the sender by return e-mail and delete it from your mailbox. Thank you.
