On Fri, Jun 07, 2019 at 09:17:29AM -0300, Jason Gunthorpe wrote: > On Fri, Jun 07, 2019 at 12:36:36PM +0200, Jan Kara wrote: > > > Because the pins would be invisible to sysadmin from that point on. > > It is not invisible, it just shows up in a rdma specific kernel > interface. You have to use rdma netlink to see the kernel object > holding this pin. > > If this visibility is the main sticking point I suggest just enhancing > the existing MR reporting to include the file info for current GUP > pins and teaching lsof to collect information from there as well so it > is easy to use. > > If the ownership of the lease transfers to the MR, and we report that > ownership to userspace in a way lsof can find, then I think all the > concerns that have been raised are met, right? I was contemplating some new lsof feature yesterday. But what I don't think we want is sysadmins to have multiple tools for multiple subsystems. Or even have to teach lsof something new for every potential new subsystem user of GUP pins. I was thinking more along the lines of reporting files which have GUP pins on them directly somewhere (dare I say procfs?) and teaching lsof to report that information. That would cover any subsystem which does a longterm pin. > > > ugly to live so we have to come up with something better. The best I can > > currently come up with is to have a method associated with the lease that > > would invalidate the RDMA context that holds the pins in the same way that > > a file close would do it. > > This is back to requiring all RDMA HW to have some new behavior they > currently don't have.. > > The main objection to the current ODP & DAX solution is that very > little HW can actually implement it, having the alternative still > require HW support doesn't seem like progress. > > I think we will eventually start seein some HW be able to do this > invalidation, but it won't be universal, and I'd rather leave it > optional, for recovery from truely catastrophic errors (ie my DAX is > on fire, I need to unplug it). Agreed. I think software wise there is not much some of the devices can do with such an "invalidate". Ira
