Taking a step back, I was a little surprised that dma_map_single
successfully returned an
iommu address for the pci bar address passed into it during my initial
experiment...
Internally, dma_map_single calls virt_to_page() to translate the
"virtual address" into a
page and intel_map page then calls page_to_phys() to convert the page to
a dma_addr_t.
The virt_to_page and page_to_phys routines don't appear to do any
validation and just
uses arithmetic to do the conversions.
the pci bar address (0x383c70e51578) does fall into a valid VA range in
x86_64 so
it could conceivably be a valid VA. So I tried an virtual address
inside the VA hole
and and it too returned without any errors.
virt_to_page(0x800000000000) -> 0xffffede000000000
page_to_phys(0xffffede000000000) -> 0xf80000000000
In arch/x86/include/asm/page.h, there is the following comment in regards to
validating the virtual address.
/*
* virt_to_page(kaddr) returns a valid pointer if and only if
* virt_addr_valid(kaddr) returns true.
*/
#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT)
So it looks like the validation by virt_addr_valid was somehow dropped
from the
virt_to_page code path. Does anyone have any ideas what happended to it?
Kit
(Resending, earlier message tagged as containing html subpart)
On 08/13/2018 08:21 AM, Kit Chow wrote:
On 08/13/2018 07:59 AM, Robin Murphy wrote:
On 13/08/18 15:23, Kit Chow wrote:
On 08/10/2018 07:10 PM, Logan Gunthorpe wrote:
On 10/08/18 06:53 PM, Kit Chow wrote:
I was able to finally succeed in doing the dma transfers over ioat
only
when prot has DMA_PTE_WRITE set by setting the direction to either
DMA_FROM_DEVICE or DMA_BIDIRECTIONAL. Any ideas if the prot settings
need to be changed? Are there any bad side effects if I used
DMA_BIDIRECTIONAL?
Good to hear it. Without digging into the direction much all I can say
is that it can sometimes be very confusing what the direction is.
Adding
another PCI device just adds to the confusion.
Yep, confusing :).
======================= =============================================
DMA_NONE no direction (used for debugging)
DMA_TO_DEVICE data is going from the memory to the device
DMA_FROM_DEVICE data is coming from the device to the memory
DMA_BIDIRECTIONAL direction isn't known
======================= =============================================
I believe, the direction should be from the IOAT's point of view.
So if
the IOAT is writing to the BAR you'd set DMA_FROM_DEVICE (ie. data is
coming from the IOAT) and if it's reading you'd set DMA_TO_DEVICE (ie.
data is going to the IOAT).
It would certainly seem like DMA_TO_DEVICE would be the proper
choice; IOAT is the plumbing to move host data (memory) to the bar
address (device).
Except that the "device" in question is the IOAT itself (more
generally, it means the device represented by the first argument to
dma_map_*() - the one actually emitting the reads and writes). The
context of a DMA API call is the individual mapping in question, not
whatever overall operation it may be part of - your example already
involves two separate mappings: one "from" system memory "to" the DMA
engine, and one "from" the DMA engine "to" PCI BAR memory.
OK, that makes sense. The middleman (aka DMA engine device) is the
key in the to/from puzzle. Thanks!
Note that the DMA API's dma_direction is also distinct from the
dmaengine API's dma_transfer_direction, and there's plenty of fun to
be had mapping between the two - see pl330.c or rcar-dmac.c for other
examples of dma_map_resource() for slave devices - no guarantees that
those implementations are entirely correct (especially the one I
did!), but in practice they do make the "DMA engine behind an IOMMU"
case work for UARTs and similar straightforward slaves.
Will go with what works and set DMA_FROM_DEVICE.
In ntb_async_tx_submit, does the direction used for the dma_map
routines for the src and dest addresses need to be consistent?
In general, the mappings of source and destination addresses would
typically have opposite directions as above, unless they're both
bidirectional.
And does the direction setting for the dmaengine_unmap_data have to
be consistent with the direction used in dma_map_*?
Yes, the arguments to an unmap are expected to match whatever was
passed to the corresponding map call. CONFIG_DMA_API_DEBUG should
help catch any mishaps.
Robin.
BTW, dmaengine_unmap routine only calls dma_unmap_page. Should it
keep track of the dma_map routine used and call the corresponding
dma_unmap routine? In the case of the intel iommu, it doesn't matter.
Thanks
Kit
Using DMA_BIDIRECTIONAL just forgoes any hardware security /
protection
that the buffer would have in terms of direction. Generally it's good
practice to use the strictest direction you can.
Given that using the pci bar address as is without getting an iommu
address results in the same "PTE Write access" error, I wonder if
there
is some internal 'prot' associated with the non-translated pci bar
address that just needs to be tweaked to include DMA_PTE_WRITE???
No, I don't think so. The 'prot' will be a property of the IOMMU. Not
having an entry is probably just the same (from the perspective of the
error you see) as only having an entry for reading.
Logan
_______________________________________________
iommu mailing list
iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linuxfoundation.org/mailman/listinfo/iommu
