For shared memory of type SHM_HUGETLB, hugetlb pages are reserved in shmget() call. If SHM_NORESERVE flags is specified then the hugetlb pages are not reserved. However when the shared memory is attached with the shmat() call the hugetlb pages are getting reserved incorrectly for SHM_HUGETLB shared memory created with SHM_NORESERVE. Ensure that the hugetlb pages are no reserved for SHM_HUGETLB shared memory in the shmat() call. Signed-off-by: Prakash Sangappa <prakash.sangappa@xxxxxxxxxx> --- fs/hugetlbfs/inode.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index f757d4f..93cafd2 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -141,7 +141,13 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma) file_accessed(file); ret = -ENOMEM; - if (!hugetlb_reserve_pages(inode, + + /* + * for SHM_HUGETLB, the pages are reserved in the shmget() call so skip + * reserving here. Note only for SHM hugetlbfs file, the inode + * flag S_PRIVATE is set. + */ + if (!(inode->i_flags & S_PRIVATE) && !hugetlb_reserve_pages(inode, vma->vm_pgoff >> huge_page_order(h), len >> huge_page_shift(h), vma, vma->vm_flags)) -- 2.7.4
