The patch titled Subject: mm: mlock: check against vma for actual mlock() size has been added to the -mm tree. Its filename is mm-mlock-check-against-vma-for-actual-mlock-size.patch This patch should soon appear at http://ozlabs.org/~akpm/mmots/broken-out/mm-mlock-check-against-vma-for-actual-mlock-size.patch and later at http://ozlabs.org/~akpm/mmotm/broken-out/mm-mlock-check-against-vma-for-actual-mlock-size.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/SubmitChecklist when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: Simon Guo <wei.guo.simon@xxxxxxxxx> Subject: mm: mlock: check against vma for actual mlock() size In do_mlock(), the check against locked memory limitation has a hole which will fail following cases at step 3): 1) User has a memory chunk from addressA with 50k, and user mem lock rlimit is 64k. 2) mlock(addressA, 30k) 3) mlock(addressA, 40k) The 3rd step should have been allowed since the 40k request is intersected with the previous 30k at step 2), and the 3rd step is actually for mlock on the extra 10k memory. This patch checks vma to caculate the actual "new" mlock size, if necessary, and ajust the logic to fix this issue. Link: http://lkml.kernel.org/r/1472554781-9835-2-git-send-email-wei.guo.simon@xxxxxxxxx Signed-off-by: Simon Guo <wei.guo.simon@xxxxxxxxx> Cc: Alexey Klimov <klimov.linux@xxxxxxxxx> Cc: Eric B Munson <emunson@xxxxxxxxxx> Cc: Geert Uytterhoeven <geert@xxxxxxxxxxxxxx> Cc: "Kirill A. Shutemov" <kirill.shutemov@xxxxxxxxxxxxxxx> Cc: Mel Gorman <mgorman@xxxxxxxxxxxxxxxxxxx> Cc: Michal Hocko <mhocko@xxxxxxxx> Cc: Shuah Khan <shuah@xxxxxxxxxx> Cc: Simon Guo <wei.guo.simon@xxxxxxxxx> Cc: Thierry Reding <treding@xxxxxxxxxx> Cc: Vlastimil Babka <vbabka@xxxxxxx> Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> --- mm/mlock.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff -puN mm/mlock.c~mm-mlock-check-against-vma-for-actual-mlock-size mm/mlock.c --- a/mm/mlock.c~mm-mlock-check-against-vma-for-actual-mlock-size +++ a/mm/mlock.c @@ -617,6 +617,43 @@ static int apply_vma_lock_flags(unsigned return error; } +/* + * Go through vma areas and sum size of mlocked + * vma pages, as return value. + * Note deferred memory locking case(mlock2(,,MLOCK_ONFAULT) + * is also counted. + * Return value: previously mlocked page counts + */ +static int count_mm_mlocked_page_nr(struct mm_struct *mm, + unsigned long start, size_t len) +{ + struct vm_area_struct *vma; + int count = 0; + + if (mm == NULL) + mm = current->mm; + + vma = find_vma(mm, start); + if (vma == NULL) + vma = mm->mmap; + + for (; vma ; vma = vma->vm_next) { + if (start + len <= vma->vm_start) + break; + if (vma->vm_flags && VM_LOCKED) { + if (start > vma->vm_start) + count -= (start - vma->vm_start); + if (start + len < vma->vm_end) { + count += start + len - vma->vm_start; + break; + } + count += vma->vm_end - vma->vm_start; + } + } + + return (PAGE_ALIGN(count) >> PAGE_SHIFT); +} + static __must_check int do_mlock(unsigned long start, size_t len, vm_flags_t flags) { unsigned long locked; @@ -639,6 +676,18 @@ static __must_check int do_mlock(unsigne return -EINTR; locked += current->mm->locked_vm; + if ((locked > lock_limit) && (!capable(CAP_IPC_LOCK))) { + /* + * It is possible that the regions requested + * intersect with previously mlocked areas, + * that part area in "mm->locked_vm" should + * not be counted to new mlock increment + * count. So check and adjust locked count + * if necessary. + */ + locked -= count_mm_mlocked_page_nr(current->mm, + start, len); + } /* check against resource limits */ if ((locked <= lock_limit) || capable(CAP_IPC_LOCK)) _ Patches currently in -mm which might be from wei.guo.simon@xxxxxxxxx are mm-mlock-check-against-vma-for-actual-mlock-size.patch mm-mlock-avoid-increase-mm-locked_vm-on-mlock-when-already-mlock2mlock_onfault.patch selftest-split-mlock2_-funcs-into-separate-mlock2h.patch selftests-vm-add-test-for-mlock-when-areas-are-intersected.patch -- To unsubscribe from this list: send the line "unsubscribe mm-commits" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html