> > On Mon, Feb 11, 2019 at 2:07 PM Jason Gunthorpe <jgg@xxxxxxxx> wrote: > > > > On Mon, Feb 11, 2019 at 01:52:38PM -0800, Ira Weiny wrote: > > > On Mon, Feb 11, 2019 at 01:39:12PM -0800, John Hubbard wrote: > > > > On 2/11/19 1:26 PM, Ira Weiny wrote: > > > > > On Mon, Feb 11, 2019 at 01:13:56PM -0800, John Hubbard wrote: > > > > >> On 2/11/19 12:39 PM, Jason Gunthorpe wrote: > > > > >>> On Mon, Feb 11, 2019 at 12:16:42PM -0800, ira.weiny@xxxxxxxxx > wrote: > > > > >>>> From: Ira Weiny <ira.weiny@xxxxxxxxx> > > > > >> [...] > > > > >> It seems to me that the longterm vs. short-term is of questionable > value. > > > > > > > > > > This is exactly why I did not post this before. I've been > > > > > waiting our other discussions on how GUP pins are going to be > > > > > handled to play out. But with the netdev thread today[1] it > > > > > seems like we need to make sure we have a "safe" fast variant > > > > > for a while. Introducing FOLL_LONGTERM seemed like the cleanest > > > > > way to do that even if we will not need the distinction in the > > > > > future... :-( > > > > > > > > Yes, I agree. Below... > > > > > > > > > [...] > > > > > This is also why I did not change the get_user_pages_longterm > > > > > because we could be ripping this all out by the end of the > > > > > year... (I hope. :-) > > > > > > > > > > So while this does "pollute" the GUP family of calls I'm hoping > > > > > it is not forever. > > > > > > > > > > Ira > > > > > > > > > > [1] https://lkml.org/lkml/2019/2/11/1789 > > > > > > > > > > > > > Yes, and to be clear, I think your patchset here is fine. It is > > > > easy to find the FOLL_LONGTERM callers if and when we want to > > > > change anything. I just think also it's appopriate to go a bit further, and > use FOLL_LONGTERM all by itself. > > > > > > > > That's because in either design outcome, it's better that way: > > > > > > > > is just right. The gup API already has _fast and non-fast > > > > variants, and once you get past a couple, you end up with a > > > > multiplication of names that really work better as flags. We're there. > > > > > > > > the _longterm API variants. > > > > > > Fair enough. But to do that correctly I think we will need to convert > > > get_user_pages_fast() to use flags as well. I have a version of > > > this series which includes a patch does this, but the patch touched > > > a lot of subsystems and a couple of different architectures...[1] > > > > I think this should be done anyhow, it is trouble the two basically > > identical interfaces have different signatures. This already caused a > > bug in vfio.. > > > > I also wonder if someone should think about making fast into a flag > > too.. > > > > But I'm not sure when fast should be used vs when it shouldn't :( > > Effectively fast should always be used just in case the user cares about > performance. It's just that it may fail and need to fall back to requiring the > vma. > > Personally I thought RDMA memory registration is a one-time / upfront slow > path so that non-fast-GUP is tolerable. > > The workloads that *need* it are O_DIRECT users that can't tolerate a vma > lookup on every I/O. There are some users who need to [un]register memory more often. While not in the strict fast path these users would like the registrations to occur as fast as possible. I don't personally have the results but our OPA team did do performance tests on the GUP vs GUP fast and for the hfi1 case fast was better. I don't have any reason to believe that regular RDMA users would not also benefit. Ira
