Changelog v3: * Register mm to khugepaged in common mmap path instead of touching filesystem code (patch 8/8) suggested by Ted. * New patch 7/8 cleaned up and renamed khugepaged_enter_vma_merge() to khugepaged_enter_vma(). * Collected acked-by from Song Liu for patch 1 ~ 6. * Rebased on top of 5.18-rc1. * Excluded linux-xfs and linux-ext4 list since the series doesn't touch fs code anymore, but keep linux-fsdevel posted. v2: * Collected reviewed-by tags from Miaohe Lin. * Fixed build error for patch 4/8. The readonly FS THP relies on khugepaged to collapse THP for suitable vmas. But it is kind of "random luck" for khugepaged to see the readonly FS vmas (see report: https://lore.kernel.org/linux-mm/00f195d4-d039-3cf2-d3a1-a2c88de397a0@xxxxxxx/) since currently the vmas are registered to khugepaged when: - Anon huge pmd page fault - VMA merge - MADV_HUGEPAGE - Shmem mmap If the above conditions are not met, even though khugepaged is enabled it won't see readonly FS vmas at all. MADV_HUGEPAGE could be specified explicitly to tell khugepaged to collapse this area, but when khugepaged mode is "always" it should scan suitable vmas as long as VM_NOHUGEPAGE is not set. So make sure readonly FS vmas are registered to khugepaged to make the behavior more consistent. Registering suitable vmas in common mmap path, that could cover both readonly FS vmas and shmem vmas, so removed the khugepaged calls in shmem.c. The patch 1 ~ 7 are minor bug fixes, clean up and preparation patches. The patch 8 is the real meat. Tested with khugepaged test in selftests and the testcase provided by Vlastimil Babka in https://lore.kernel.org/lkml/df3b5d1c-a36b-2c73-3e27-99e74983de3a@xxxxxxx/ by commenting out MADV_HUGEPAGE call. Yang Shi (8): sched: coredump.h: clarify the use of MMF_VM_HUGEPAGE mm: khugepaged: remove redundant check for VM_NO_KHUGEPAGED mm: khugepaged: skip DAX vma mm: thp: only regular file could be THP eligible mm: khugepaged: make khugepaged_enter() void function mm: khugepaged: move some khugepaged_* functions to khugepaged.c mm: khugepaged: introduce khugepaged_enter_vma() helper mm: mmap: register suitable readonly file vmas for khugepaged include/linux/huge_mm.h | 14 ++++++++++++ include/linux/khugepaged.h | 59 ++++++++++++--------------------------------------- include/linux/sched/coredump.h | 3 ++- kernel/fork.c | 4 +--- mm/huge_memory.c | 15 ++++--------- mm/khugepaged.c | 76 +++++++++++++++++++++++++++++++++++++----------------------------- mm/mmap.c | 14 ++++++++---- mm/shmem.c | 12 ----------- 8 files changed, 88 insertions(+), 109 deletions(-)