Hi All, I am using the patch "Multi-size THP for anonymous memory" https://lore.kernel.org/all/20231214160251.3574571-1-ryan.roberts@xxxxxxx/T/#u I enabled the mTHP using the sysfs interface : echo always >/sys/kernel/mm/transparent_hugepage/hugepages-16kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-32kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-64kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-128kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-256kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-512kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-1024kB/enabled echo always >/sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled I can see this patch allocates multi-order folio for anonymous memory. With the large order folios getting allocated I tried direct block IO using fio. fio -iodepth=1 -rw=write -ioengine=io_uring -direct=1 -bs=16K -numjobs=1 -size=16k -group_reporting -filename=/dev/nvme0n1 -name=io_uring_test The fio malloced memory is allocated from a multi-order folio in function alloc_anon_folio(). Block I/O path takes the fio allocated memory and maps it in kernel in function iov_iter_extract_user_pages() As the pages are mapped using large folios, I try to see if the pages belong to same folio using page_folio(page) in function __bio_iov_iter_get_pages(). To my surprise I see that the pages belong to different folios. Feb 5 10:34:33 kernel: [244413.315660] 1603 iov_iter_extract_user_pages addr = 5593b252a000 Feb 5 10:34:33 kernel: [244413.315680] 1610 iov_iter_extract_user_pages nr_pages = 4 Feb 5 10:34:33 kernel: [244413.315700] 1291 __bio_iov_iter_get_pages page = ffffea000d4bb9c0 folio = ffffea000d4bb9c0 Feb 5 10:34:33 kernel: [244413.315749] 1291 __bio_iov_iter_get_pages page = ffffea000d796200 folio = ffffea000d796200 Feb 5 10:34:33 kernel: [244413.315796] 1291 __bio_iov_iter_get_pages page = ffffea000d796240 folio = ffffea000d796240 Feb 5 10:34:33 kernel: [244413.315852] 1291 __bio_iov_iter_get_pages page = ffffea000d7b2b80 folio = ffffea000d7b2b80 I repeat the same experiment with fio using HUGE pages fio -iodepth=1 -iomem=mmaphuge -rw=write -ioengine=io_uring -direct=1 -bs=16K -numjobs=1 -size=16k -group_reporting -filename=/dev/nvme0n1 -name=io_uring_test This time when the memory is mmapped from HUGE pages I see that pages belong to the same folio. Feb 5 10:51:50 kernel: [245450.439817] 1603 iov_iter_extract_user_pages addr = 7f66e4c00000 Feb 5 10:51:50 kernel: [245450.439825] 1610 iov_iter_extract_user_pages nr_pages = 4 Feb 5 10:51:50 kernel: [245450.439834] 1291 __bio_iov_iter_get_pages page = ffffea0005bc8000 folio = ffffea0005bc8000 Feb 5 10:51:50 kernel: [245450.439858] 1291 __bio_iov_iter_get_pages page = ffffea0005bc8040 folio = ffffea0005bc8000 Feb 5 10:51:50 kernel: [245450.439880] 1291 __bio_iov_iter_get_pages page = ffffea0005bc8080 folio = ffffea0005bc8000 Feb 5 10:51:50 kernel: [245450.439903] 1291 __bio_iov_iter_get_pages page = ffffea0005bc80c0 folio = ffffea0005bc8000 Please let me know if you have any clue as to why the pages for malloced memory of fio don't belong to the same folio. -- Kundan Kumar
