On Fri 18-09-20 21:01:53, Jason Gunthorpe wrote: > On Fri, Sep 18, 2020 at 02:06:23PM -0700, John Hubbard wrote: > > On 9/18/20 1:40 PM, Peter Xu wrote: > > > On Fri, Sep 18, 2020 at 02:32:40PM -0300, Jason Gunthorpe wrote: > > > > On Fri, Sep 18, 2020 at 12:40:32PM -0400, Peter Xu wrote: > > > > > > > > > Firstly in the draft patch mm->has_pinned is introduced and it's written to 1 > > > > > as long as FOLL_GUP is called once. It's never reset after set. > > > > > > > > Worth thinking about also adding FOLL_LONGTERM here, at last as long > > > > as it is not a counter. That further limits the impact. > > > > > > But theoritically we should also trigger COW here for pages even with PIN && > > > !LONGTERM, am I right? Assuming that FOLL_PIN is already a corner case. > > > > > > > This note, plus Linus' comment about "I'm a normal process, I've never > > done any special rdma page pinning", has me a little worried. Because > > page_maybe_dma_pinned() is counting both short- and long-term pins, > > actually. And that includes O_DIRECT callers. > > > > O_DIRECT pins are short-term, and RDMA systems are long-term (and should > > be setting FOLL_LONGTERM). But there's no way right now to discern > > between them, once the initial pin_user_pages*() call is complete. All > > we can do today is to count the number of FOLL_PIN calls, not the number > > of FOLL_PIN | FOLL_LONGTERM calls. > > My thinking is to hit this issue you have to already be doing > FOLL_LONGTERM, and if some driver hasn't been properly marked and > regresses, the fix is to mark it. > > Remember, this use case requires the pin to extend after a system > call, past another fork() system call, and still have data-coherence. > > IMHO that can only happen in the FOLL_LONGTERM case as it inhernetly > means the lifetime of the pin is being controlled by userspace, not by > the kernel. Otherwise userspace could not cause new DMA touches after > fork. I agree that the new aggressive COW behavior is probably causing issues only for FOLL_LONGTERM users. That being said it would be nice if even ordinary threaded FOLL_PIN users would not have to be that careful about fork(2) and possible data loss due to COW - we had certainly reports of O_DIRECT IO loosing data due to fork(2) and COW exactly because it is very subtle how it behaves... But as I wrote above this is not urgent since that problematic behavior exists since the beginning of O_DIRECT IO in Linux. Honza -- Jan Kara <jack@xxxxxxxx> SUSE Labs, CR
