On Wed, Dec 12, 2018 at 04:18:33PM -0800, Dan Williams wrote: > On Wed, Dec 12, 2018 at 4:01 PM Jerome Glisse <jglisse@xxxxxxxxxx> wrote: > > > > On Wed, Dec 12, 2018 at 04:37:03PM -0700, Jason Gunthorpe wrote: > > > On Wed, Dec 12, 2018 at 04:53:49PM -0500, Jerome Glisse wrote: > > > > > Almost, we need some safety around assuming that DMA is complete the > > > > > page, so the notification would need to go all to way to userspace > > > > > with something like a file lease notification. It would also need to > > > > > be backstopped by an IOMMU in the case where the hardware does not / > > > > > can not stop in-flight DMA. > > > > > > > > You can always reprogram the hardware right away it will redirect > > > > any dma to the crappy page. > > > > > > That causes silent data corruption for RDMA users - we can't do that. > > > > > > The only way out for current hardware is to forcibly terminate the > > > RDMA activity somehow (and I'm not even sure this is possible, at > > > least it would be driver specific) > > > > > > Even the IOMMU idea probably doesn't work, I doubt all current > > > hardware can handle a PCI-E error TLP properly. > > > > What i saying is reprogram hardware to crappy page ie valid page > > dma map but that just has random content as a last resort to allow > > filesystem to reuse block. So their should be no PCIE error unless > > hardware freak out to see its page table reprogram randomly. > > Hardware has a hard enough time stopping I/O to existing page let > alone switching to a new one in the middle of a transaction. This is a > non-starter, but it's also a non-concern because the bulk of DMA is > transient. For non-transient DMA there is a usually a registration > phase where the capability to support revocation can be validated, On many GPUs you can do that, it is hardware dependant and you have steps to take but it is something you can do (and GPU can do continuous DMA traffic have they have threads running that can do continuous memory access). So i assume that other hardware can do it too. Any revocation mechanism gonna be device/sub-system specific so it would probably be better to talk case by case and see what we can do. Like i said posted patches to remove GUP from GPUs driver, i am working on improving some core code to make those patches even simpler and i will keep pushing for that in subsystem i know. Maybe we should grep for GUP in drivers/ and start discussion within each sub-system to see what can be done within each. If any common pattern emerge we can draw up common plans for those at least. Cheers, Jérôme
