On Mon, Sep 14, 2020 at 05:15:15PM -0400, Peter Xu wrote: > On Mon, Sep 14, 2020 at 02:34:36PM -0400, Peter Xu wrote: > > On Mon, Sep 14, 2020 at 10:32:11AM -0700, Linus Torvalds wrote: > > > On Mon, Sep 14, 2020 at 7:38 AM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > > > > > > > I don't have a detailed explanation right now, but this patch appears > > > > to be causing a regression where RDMA subsystem tests fail. Tests > > > > return to normal when this patch is reverted. > > > > > > > > It kind of looks like the process is not seeing DMA'd data to a > > > > pin_user_pages()? > > > > > > I'm a nincompoop. I actually _talked_ to Hugh Dickins about this when > > > he raised concerns, and I dismissed his concerns with "but PAGE_PIN is > > > special". > > > > > > As usual, Hugh was right. Page pinning certainly _is_ special, but > > > it's not that different from the regular GUP code. > > > > > > But in the meantime, I have a lovely confirmation from the kernel test > > > robot, saying that commit 09854ba94c results in a > > > "vm-scalability.throughput 31.4% improvement", which was what I was > > > hoping for - the complexity wasn't just complexity, it was active > > > badness due to the page locking horrors. > > > > > > I think what we want to do is basically do the "early COW", but only > > > do it for FOLL_PIN (and not turn them into writes for anything but the > > > COW code). So basically redo the "enforced COW mechanism", but rather > > > than do it for everything, now do it only for FOLL_PIN, and only in > > > that COW path. > > > > > > Peter - any chance you can look at this? I'm still looking at the page > > > lock fairness performance regression, although I now think I have a > > > test patch for Phoronix to test out. > > > > Sure, I'll try to prepare something like that and share it shortly. > > Jason, would you please try the attached patch to see whether it unbreaks the > rdma test? Thanks! Sure, I'll get back to you > Fast gup is not affected by this because it is never used with FOLL_PIN. ? What is pin_user_pages_fast() then? That is the API the failing test is using. > Note: hugetlbfs is not considered throughout this patch, because it's missing > some required bits after all (like proper setting of FOLL_COW when page fault > retries). Considering we may want to unbreak RDMA tests even during the rcs, > this patch only fixes the non-hugetlbfs cases. THPs should still be in count. People do RDMA with hugetlbfs too. Just as an aside, the RDMA stuff is also supposed to set MADV_DONTFORK on these regions, so I'm a bit puzzled what is happening here. Jason
