Hey, On 22/11/16 11:59 AM, Serguei Sagalovitch wrote: > - How well we will be able to handle case when we need to "move"/"evict" > memory/data to the new location so CPU pointer should point to the > new physical location/address > (and may be not in PCI device memory at all)? IMO any memory that has been registered for a P2P transaction should be locked from being evicted. So if there's a get_user_pages call it needs to be pinned until the put_page. The main issue being with the RDMA case: handling an eviction when a chunk of memory has been registered as an MR would be very tricky. The MR may be relied upon by another host and the kernel would have to inform user-space the MR was invalid then user-space would have to tell the remote application. This seems like a lot of burden to place on applications and may be subject to timing issues. Either that or all RDMA applications need to be written with the assumption that their target memory could go away at any time. More generally, if you tell one PCI device to do a DMA transfer to another PCI device's BAR space, and the target memory gets evicted then DMA transaction needs to be aborted which means every driver doing the transfer would need special support for this. If the memory can be relied on to not be evicted than existing drivers should work unmodified (ie O_DIRECT to/from an NVMe card would just work). I feel the better approach is to pin memory subject to P2P transactions as is typically done with DMA transfers to main memory. Logan -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
