On Wed, Apr 13, 2022 at 12:10:51PM -0700, Andrew Morton wrote: > On Wed, 13 Apr 2022 22:47:48 +0800 Muchun Song <songmuchun@xxxxxxxxxxxxx> wrote: > > > We must add hugetlb_free_vmemmap=on (or "off") to the boot cmdline and > > reboot the server to enable or disable the feature of optimizing vmemmap > > pages associated with HugeTLB pages. However, rebooting usually takes a > > long time. So add a sysctl to enable or disable the feature at runtime > > without rebooting. > > Do we really need this feature? Really? What's the use case and what > is the end-user value? > We know that this feature is disabled by default without passing "hugetlb_free_vmemmap=on" to the boot cmdline. When we (ByteDance) deliver the servers to the users who want to enable this feature, they have to configure the grub (change boot cmdline) and reboot the servers, whereas rebooting usually takes a long time (we have thousands of servers). It's a very bad experience for the users. So we need a approach to enable this feature after rebooting. This is a use case in our practical environment. > Presumably CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP worsens things for some > setups/workloads? Please tell us much more about that. What is the > magnitude of the deoptimization? > Most workloads are allocated HugeTLB from the pool not the buddy allocator, which are the users of this feature. However if the use case is that HugeTLB pages are allocated 'on the fly' instead of being pulled from the HugeTLB pool, those workloads would be affected with this feature enabled. Those workloads could be identified by the characteristics of they never explicitly allocating huge pages with 'nr_hugepages' but only set 'nr_overcommit_hugepages' and then let the pages be allocated from the buddy allocator at fault time. I don't know what the specific workload is. But I can confirm it is a real use case from the commit 099730d67417. For those workloads, the page fault time could be ~2x slower than before. I suspect those users want to disable this feature if the system has enabled this before if they don't think the memory savings benefit is enough to make up for the performance drop. Do you think it's ok if I put those information in the commit log or vm.rst? Thanks.
