In vma_adjust, the current code grabs i_mmap_mutex before calling vma_adjust_trans_huge. This used to be fine until huge page in page cache comes in. The problem is the underlying function split_file_huge_page will also grab the i_mmap_mutex before splitting the huge page in page cache. Obviously this is causing deadlock situation. This fix is to move the vma_adjust_trans_huge before grab the lock for file, the same as what the function is currently doing for anonymous memory. Tested, everything works fine so far. Signed-off-by: Ning Qu <quning@xxxxxxxxxx> --- mm/mmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 519ce78..accf1b3 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -750,6 +750,8 @@ again: remove_next = 1 + (end > next->vm_end); } } + vma_adjust_trans_huge(vma, start, end, adjust_next); + if (file) { mapping = file->f_mapping; if (!(vma->vm_flags & VM_NONLINEAR)) { @@ -773,8 +775,6 @@ again: remove_next = 1 + (end > next->vm_end); } } - vma_adjust_trans_huge(vma, start, end, adjust_next); - anon_vma = vma->anon_vma; if (!anon_vma && adjust_next) anon_vma = next->anon_vma; -- 1.8.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx"> email@xxxxxxxxx </a>