On 11.02.22 13:23, Aneesh Kumar K.V wrote:
> David Hildenbrand <david@xxxxxxxxxx> writes:
>
>> On 11.02.22 10:16, Aneesh Kumar K V wrote:
>>> On 2/11/22 14:00, David Hildenbrand wrote:
>>>> On 11.02.22 07:52, Aneesh Kumar K.V wrote:
>>>>> commit: d9c234005227 ("Do not depend on MAX_ORDER when grouping pages by mobility")
>>>>> introduced pageblock_order which will be used to group pages better.
>>>>> The kernel now groups pages based on the value of HPAGE_SHIFT. Hence HPAGE_SHIFT
>>>>> should be set before we call set_pageblock_order.
>>>>>
>>>>> set_pageblock_order happens early in the boot and default hugetlb page size
>>>>> should be initialized before that to compute the right pageblock_order value.
>>>>>
>>>>> Currently, default hugetlbe page size is set via arch_initcalls which happens
>>>>> late in the boot as shown via the below callstack:
>>>>>
>>>>> [c000000007383b10] [c000000001289328] hugetlbpage_init+0x2b8/0x2f8
>>>>> [c000000007383bc0] [c0000000012749e4] do_one_initcall+0x14c/0x320
>>>>> [c000000007383c90] [c00000000127505c] kernel_init_freeable+0x410/0x4e8
>>>>> [c000000007383da0] [c000000000012664] kernel_init+0x30/0x15c
>>>>> [c000000007383e10] [c00000000000cf14] ret_from_kernel_thread+0x5c/0x64
>>>>>
>>>>> and the pageblock_order initialization is done early during the boot.
>>>>>
>>>>> [c0000000018bfc80] [c0000000012ae120] set_pageblock_order+0x50/0x64
>>>>> [c0000000018bfca0] [c0000000012b3d94] sparse_init+0x188/0x268
>>>>> [c0000000018bfd60] [c000000001288bfc] initmem_init+0x28c/0x328
>>>>> [c0000000018bfe50] [c00000000127b370] setup_arch+0x410/0x480
>>>>> [c0000000018bfed0] [c00000000127401c] start_kernel+0xb8/0x934
>>>>> [c0000000018bff90] [c00000000000d984] start_here_common+0x1c/0x98
>>>>>
>>>>> delaying default hugetlb page size initialization implies the kernel will
>>>>> initialize pageblock_order to (MAX_ORDER - 1) which is not an optimal
>>>>> value for mobility grouping. IIUC we always had this issue. But it was not
>>>>> a problem for hash translation mode because (MAX_ORDER - 1) is the same as
>>>>> HUGETLB_PAGE_ORDER (8) in the case of hash (16MB). With radix,
>>>>> HUGETLB_PAGE_ORDER will be 5 (2M size) and hence pageblock_order should be
>>>>> 5 instead of 8.
>>>>
>>>>
>>>> A related question: Can we on ppc still have pageblock_order > MAX_ORDER
>>>> - 1? We have some code for that and I am not so sure if we really need that.
>>>>
>>>
>>> I also have been wondering about the same. On book3s64 I don't think we
>>> need that support for both 64K and 4K page size because with hash
>>> hugetlb size is MAX_ORDER -1. (16MB hugepage size)
>>>
>>> I am not sure about the 256K page support. Christophe may be able to
>>> answer that.
>>>
>>> For the gigantic hugepage support we depend on cma based allocation or
>>> firmware reservation. So I am not sure why we ever considered pageblock
>>> > MAX_ORDER -1 scenario. If you have pointers w.r.t why that was ever
>>> needed, I could double-check whether ppc64 is still dependent on that.
>>
>> commit dc78327c0ea7da5186d8cbc1647bd6088c5c9fa5
>> Author: Michal Nazarewicz <mina86@xxxxxxxxxx>
>> Date: Wed Jul 2 15:22:35 2014 -0700
>>
>> mm: page_alloc: fix CMA area initialisation when pageblock > MAX_ORDER
>>
>> indicates that at least arm64 used to have cases for that as well.
>>
>> However, nowadays with ARM64_64K_PAGES we have FORCE_MAX_ZONEORDER=14 as
>> default, corresponding to 512MiB.
>>
>> So I'm not sure if this is something worth supporting. If you want
>> somewhat reliable gigantic pages, use CMA or preallocate them during boot.
>>
>> --
>> Thanks,
>>
>> David / dhildenb
>
> I could build a kernel with FORCE_MAX_ZONEORDER=8 and pageblock_order =
> 8. We need to disable THP for such a kernel to boot, because THP do
> check for PMD_ORDER < MAX_ORDER. I was able to boot that kernel on a
> virtualized platform, but then gigantic_page_runtime_supported is not
> supported on such config with hash translation.
>
I'm currently playing with the idea of the following (uncompiled,untested):
