SGX memory is managed outside the core mm. It doesn't have a 'struct page' and get_user_pages() doesn't work on it. Its VMAs are marked with VM_IO. So, none of the existing methods for avoiding page faults work on SGX memory. Add f_op->populate() to overcome this issue: int (*populate)(struct file *, unsigned long start, unsigned long end); Then in populate_vma_page_range(), allow it to be used in the place of get_user_pages() for memory that falls outside of its scope. Signed-off-by: Jarkko Sakkinen <jarkko@xxxxxxxxxx> --- v5: * In v4, one diff was left out of staging area in __mm_populate(). It was unintentional to remove the conditional statement. v4: * Reimplement based on Dave's suggestion: https://lore.kernel.org/linux-sgx/c3083144-bfc1-3260-164c-e59b2d110df8@xxxxxxxxx/ * Copy the text from the suggestion as part of the commit message (and cover letter). v3: - if (!ret && do_populate && file->f_op->populate) + if (!ret && do_populate && file->f_op->populate && + !!(vma->vm_flags & (VM_IO | VM_PFNMAP))) (reported by Matthew Wilcox) v2: - if (!ret && do_populate) + if (!ret && do_populate && file->f_op->populate) (reported by Jan Harkes) --- include/linux/fs.h | 1 + mm/gup.c | 11 ++++++++--- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index e2d892b201b0..54151af88ee0 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1993,6 +1993,7 @@ struct file_operations { long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); long (*compat_ioctl) (struct file *, unsigned int, unsigned long); int (*mmap) (struct file *, struct vm_area_struct *); + int (*populate)(struct file *, unsigned long start, unsigned long end); unsigned long mmap_supported_flags; int (*open) (struct inode *, struct file *); int (*flush) (struct file *, fl_owner_t id); diff --git a/mm/gup.c b/mm/gup.c index a9d4d724aef7..1f3a1d0b6e0d 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1519,8 +1519,11 @@ long populate_vma_page_range(struct vm_area_struct *vma, * We made sure addr is within a VMA, so the following will * not result in a stack expansion that recurses back here. */ - return __get_user_pages(mm, start, nr_pages, gup_flags, - NULL, NULL, locked); + if ((vma->vm_flags & (VM_IO | VM_PFNMAP)) && vma->vm_file->f_op->populate) + return vma->vm_file->f_op->populate(vma->vm_file, start, end); + else + return __get_user_pages(mm, start, nr_pages, gup_flags, + NULL, NULL, locked); } /* @@ -1598,6 +1601,7 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors) struct vm_area_struct *vma = NULL; int locked = 0; long ret = 0; + bool is_io; end = start + len; @@ -1619,7 +1623,8 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors) * range with the first VMA. Also, skip undesirable VMA types. */ nend = min(end, vma->vm_end); - if (vma->vm_flags & (VM_IO | VM_PFNMAP)) + is_io = !!(vma->vm_flags & (VM_IO | VM_PFNMAP)); + if (is_io && !(is_io && vma->vm_file->f_op->populate)) continue; if (nstart < vma->vm_start) nstart = vma->vm_start; -- 2.35.1
