The mmap_region() function is somewhat terrifying, with spaghetti-like control flow and numerous means by which issues can arise and incomplete state, memory leaks and other unpleasantness can occur. A large amount of the complexity arises from trying to handle errors late in the process of mapping a VMA, which forms the basis of recently observed issues with resource leaks and observable inconsistent state. This series goes to great lengths to simplify how mmap_region() works and to avoid unwinding errors late on in the process of setting up the VMA for the new mapping, and equally avoids such operations occurring while the VMA is in an inconsistent state. The first four patches are intended for backporting to correct the possibility of people encountering corrupted state while invoking mmap() which is otherwise at risk of happening. After this we go further, refactoring the code, placing it in mm/vma.c in order to make it eventually userland testable, and significantly simplifying the logic to avoid this issue arising in future. v2: * Marked first 4 patches as hotfixes, the rest as not. * Improved comment in vma_close() as per Vlastiml. * Updated hole byte count as per Jann. * Updated comment in map_deny_write_exec() as per Jann. * Dropped unnecessary vma_iter_free() as per Vlastmil, Liam. * Corrected vms_abort_munmap_vmas() mistaken assumption about nr_pages as per Vlasitmil. * Changed order of initial checks in mmap_region() to avoid user-visible side effects as per Vmastlil, Liam. * Corrected silly incorrect use of vma field. * Various style corrects as per Liam. * Fix horrid mistake with merge VMA, reworked the logic to avoid that nonsense altogether. * Add fields to map state rather than using vmg fields to avoid confusion/risk of vmg state changing breaking things. * Replaced last commit removing merge retry with one that retries the merge, only sanely. v1: https://lore.kernel.org/all/cover.1729628198.git.lorenzo.stoakes@xxxxxxxxxx/ Lorenzo Stoakes (8): mm: avoid unsafe VMA hook invocation when error arises on mmap hook mm: unconditionally close VMAs on error mm: refactor map_deny_write_exec() mm: resolve faulty mmap_region() error path behaviour tools: testing: add additional vma_internal.h stubs mm: isolate mmap internal logic to mm/vma.c mm: refactor __mmap_region() mm: defer second attempt at merge on mmap() include/linux/mman.h | 21 +- mm/internal.h | 45 ++++ mm/mmap.c | 262 ++----------------- mm/mprotect.c | 2 +- mm/nommu.c | 7 +- mm/vma.c | 435 ++++++++++++++++++++++++++++++- mm/vma.h | 103 +------- mm/vma_internal.h | 5 + tools/testing/vma/vma_internal.h | 115 +++++++- 9 files changed, 634 insertions(+), 361 deletions(-) -- 2.47.0