The patch titled
Subject: mm: fix a huge pud insertion race during faulting
has been added to the -mm tree. Its filename is
mm-fix-a-huge-pud-insertion-race-during-faulting.patch
This patch should soon appear at
http://ozlabs.org/~akpm/mmots/broken-out/mm-fix-a-huge-pud-insertion-race-during-faulting.patch
and later at
http://ozlabs.org/~akpm/mmotm/broken-out/mm-fix-a-huge-pud-insertion-race-during-faulting.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Thomas Hellstrom <thellstrom@xxxxxxxxxx>
Subject: mm: fix a huge pud insertion race during faulting
A huge pud page can theoretically be faulted in racing with pmd_alloc() in
__handle_mm_fault(). That will lead to pmd_alloc() returning an invalid
pmd pointer. Fix this by adding a pud_trans_unstable() function similar
to pmd_trans_unstable() and check whether the pud is really stable before
using the pmd pointer.
Race:
Thread 1: Thread 2: Comment
create_huge_pud() Fallback - not taken.
create_huge_pud() Taken.
pmd_alloc() Returns an invalid pointer.
Link: http://lkml.kernel.org/r/20191115115808.21181-2-thomas_os@xxxxxxxxxxxx
Fixes: a00cc7d9dd93 ("mm, x86: add support for PUD-sized transparent hugepages")
Signed-off-by: Thomas Hellstrom <thellstrom@xxxxxxxxxx>
Cc: Arnd Bergmann <arnd@xxxxxxxx>
Cc: "Kirill A. Shutemov" <kirill.shutemov@xxxxxxxxxxxxxxx>
Cc: Matthew Wilcox <willy@xxxxxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
include/asm-generic/pgtable.h | 25 +++++++++++++++++++++++++
mm/memory.c | 6 ++++++
2 files changed, 31 insertions(+)
--- a/include/asm-generic/pgtable.h~mm-fix-a-huge-pud-insertion-race-during-faulting
+++ a/include/asm-generic/pgtable.h
@@ -938,6 +938,31 @@ static inline int pud_trans_huge(pud_t p
}
#endif
+/* See pmd_none_or_trans_huge_or_clear_bad for discussion. */
+static inline int pud_none_or_trans_huge_or_dev_or_clear_bad(pud_t *pud)
+{
+ pud_t pudval = READ_ONCE(*pud);
+
+ if (pud_none(pudval) || pud_trans_huge(pudval) || pud_devmap(pudval))
+ return 1;
+ if (unlikely(pud_bad(pudval))) {
+ pud_clear_bad(pud);
+ return 1;
+ }
+ return 0;
+}
+
+/* See pmd_trans_unstable for discussion. */
+static inline int pud_trans_unstable(pud_t *pud)
+{
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \
+ defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)
+ return pud_none_or_trans_huge_or_dev_or_clear_bad(pud);
+#else
+ return 0;
+#endif
+}
+
#ifndef pmd_read_atomic
static inline pmd_t pmd_read_atomic(pmd_t *pmdp)
{
--- a/mm/memory.c~mm-fix-a-huge-pud-insertion-race-during-faulting
+++ a/mm/memory.c
@@ -3936,6 +3936,7 @@ static vm_fault_t __handle_mm_fault(stru
vmf.pud = pud_alloc(mm, p4d, address);
if (!vmf.pud)
return VM_FAULT_OOM;
+retry_pud:
if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) {
ret = create_huge_pud(&vmf);
if (!(ret & VM_FAULT_FALLBACK))
@@ -3962,6 +3963,11 @@ static vm_fault_t __handle_mm_fault(stru
vmf.pmd = pmd_alloc(mm, vmf.pud, address);
if (!vmf.pmd)
return VM_FAULT_OOM;
+
+ /* Huge pud page fault raced with pmd_alloc? */
+ if (pud_trans_unstable(vmf.pud))
+ goto retry_pud;
+
if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) {
ret = create_huge_pmd(&vmf);
if (!(ret & VM_FAULT_FALLBACK))
_
Patches currently in -mm which might be from thellstrom@xxxxxxxxxx are
mm-move-the-backup-x_devmap-functions-to-asm-generic-pgtableh.patch
mm-fix-a-huge-pud-insertion-race-during-faulting.patch
