DAX has the following issues in a shared or read-only private
mmap'd file.
- mmap(MAP_POPULATE) does not pre-fault
- mlock() fails with -ENOMEM
DAX uses VM_MIXEDMAP for mmap'd files, which do not have struct
page associated with the ranges. Both MAP_POPULATE and mlock()
call __mm_populate(), which in turn calls __get_user_pages().
Because __get_user_pages() requires a valid page returned from
follow_page_mask(), MAP_POPULATE and mlock(), i.e. FOLL_POPULATE,
fail in the first page.
Change __get_user_pages() to proceed FOLL_POPULATE when the
translation is set but its page does not exist (-EFAULT), and
@pages is not requested. With that, MAP_POPULATE and mlock()
set translations to the requested range and complete successfully.
MAP_POPULATE still provides a major performance improvement to
DAX as it will avoid page faults during initial access to the
pages.
mlock() continues to set VM_LOCKED to vma and populate the range.
Since there is no struct page, the range is pinned without marking
pages mlocked.
Note, MAP_POPULATE and mlock() already work for a write-able
private mmap'd file on DAX since populate_vma_page_range() breaks
COW, which allocates page caches.
Signed-off-by: Toshi Kani <toshi.kani@xxxxxx>
---
mm/gup.c | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/mm/gup.c b/mm/gup.c
index 6297f6b..16d536f 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -490,8 +490,20 @@ retry:
}
BUG();
}
- if (IS_ERR(page))
+ if (IS_ERR(page)) {
+ /*
+ * No page may be associated with VM_MIXEDMAP. Proceed
+ * FOLL_POPULATE when the translation is set but its
+ * page does not exist (-EFAULT), and @pages is not
+ * requested by the caller.
+ */
+ if ((PTR_ERR(page) == -EFAULT) && (!pages) &&
+ (gup_flags & FOLL_POPULATE) &&
+ (vma->vm_flags & VM_MIXEDMAP))
+ goto next_page;
+
return i ? i : PTR_ERR(page);
+ }
if (pages) {
pages[i] = page;
flush_anon_page(vma, page, start);
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
