On Mon, Feb 17, 2025 at 8:14 AM Usama Arif <usamaarif642@xxxxxxxxx> wrote: > > > > On 11/02/2025 00:40, Nico Pache wrote: > > Now that we have mTHP support in khugepaged, lets add it to the > > transhuge admin guide to provide proper guidance. > > > > I think you should move this patch to the mTHP khugepaged series, and just send > THP=defer separately from mTHP khguepaged. > > > Signed-off-by: Nico Pache <npache@xxxxxxxxxx> > > --- > > Documentation/admin-guide/mm/transhuge.rst | 22 ++++++++++++++++------ > > 1 file changed, 16 insertions(+), 6 deletions(-) > > > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > > index b3b18573bbb4..99ba3763c1c4 100644 > > --- a/Documentation/admin-guide/mm/transhuge.rst > > +++ b/Documentation/admin-guide/mm/transhuge.rst > > @@ -63,7 +63,7 @@ often. > > THP can be enabled system wide or restricted to certain tasks or even > > memory ranges inside task's address space. Unless THP is completely > > disabled, there is ``khugepaged`` daemon that scans memory and > > -collapses sequences of basic pages into PMD-sized huge pages. > > +collapses sequences of basic pages into huge pages. > > > > The THP behaviour is controlled via :ref:`sysfs <thp_sysfs>` > > interface and using madvise(2) and prctl(2) system calls. > > @@ -103,8 +103,8 @@ madvise(MADV_HUGEPAGE) on their critical mmapped regions. > > Applications that would like to benefit from THPs but would still like a > > more memory conservative approach can choose 'defer'. This avoids > > inserting THPs at the page fault handler unless they are MADV_HUGEPAGE. > > -Khugepaged will then scan the mappings for potential collapses into PMD > > -sized pages. Admins using this the 'defer' setting should consider > > +Khugepaged will then scan the mappings for potential collapses into (m)THP > > +pages. Admins using this the 'defer' setting should consider > > tweaking khugepaged/max_ptes_none. The current default of 511 may > > aggressively collapse your PTEs into PMDs. Lower this value to conserve > > more memory (ie. max_ptes_none=64). > > @@ -119,11 +119,14 @@ Global THP controls > > > > Transparent Hugepage Support for anonymous memory can be entirely disabled > > (mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE > > -regions (to avoid the risk of consuming more memory resources) or enabled > > -system wide. This can be achieved per-supported-THP-size with one of:: > > +regions (to avoid the risk of consuming more memory resources), defered to > > +khugepaged, or enabled system wide. > > + > > +This can be achieved per-supported-THP-size with one of:: > > > > echo always >/sys/kernel/mm/transparent_hugepage/hugepages-<size>kB/enabled > > echo madvise >/sys/kernel/mm/transparent_hugepage/hugepages-<size>kB/enabled > > + echo defer >/sys/kernel/mm/transparent_hugepage/hugepages-<size>kB/enabled > > echo never >/sys/kernel/mm/transparent_hugepage/hugepages-<size>kB/enabled > > > > where <size> is the hugepage size being addressed, the available sizes > > @@ -155,6 +158,13 @@ hugepage sizes have enabled="never". If enabling multiple hugepage > > sizes, the kernel will select the most appropriate enabled size for a > > given allocation. > > > > +khugepaged use max_ptes_none scaled to the order of the enabled mTHP size to > > +determine collapses. When using mTHPs its recommended to set max_ptes_none low. > > +Ideally less than HPAGE_PMD_NR / 2 (255 on 4k page size). This will prevent > > +undesired "creep" behavior that leads to continously collapsing to a larger > > +mTHP size. max_ptes_shared and max_ptes_swap have no effect when collapsing to a > > +mTHP, and mTHP collapse will fail on shared or swapped out pages. > > + > > This paragraph definitely belongs in the khugepaged series, as it doesn't have anything > to do with THP=defer. > > re "Ideally less than HPAGE_PMD_NR / 2", > what if you are running on amd, and using 16K and 2M THP=always only as, thats where > the most TLB benefit is. Than this recommendation doesnt make sense? That may be correct, I believe the creep requires two adjacent mTHP levels ( ie 512kb, 1024kb) to be enabled for the issue to really present. Although with max_ptes_none=511, you will almost always satisfy the collapse request, so your 16Kb mTHPs will be promoted to PMDs. I dont believe 511 is a good default if using mTHPs. > > Also even if you have all mTHP sizes as always, shouldnt you start by collapsing to > the largest THP size first? (I haven't reviewed the khugepaged series yet, so might > be have been discussed there, I will try and review it). We do start at the largest first. The creep happens on a second pass of the PMD, not immediately in the same collapse. > > Did you see the creep behavior you mentioned in your experiments? Yes, I provided an example of how it happens here. https://lore.kernel.org/lkml/CAA1CXcDiGLD=dZpFRyAuz4TLrVZZYGp=u7=Z9Q+g9RXbf-s2nA@xxxxxxxxxxxxxx/ > > > > It's also possible to limit defrag efforts in the VM to generate > > anonymous hugepages in case they're not immediately free to madvise > > regions or to never try to defrag memory and simply fallback to regular > > @@ -318,7 +328,7 @@ Alternatively, each supported anonymous THP size can be controlled by > > passing ``thp_anon=<size>[KMG],<size>[KMG]:<state>;<size>[KMG]-<size>[KMG]:<state>``, > > where ``<size>`` is the THP size (must be a power of 2 of PAGE_SIZE and > > supported anonymous THP) and ``<state>`` is one of ``always``, ``madvise``, > > -``never`` or ``inherit``. > > +``defer``, ``never`` or ``inherit``. > > > > For example, the following will set 16K, 32K, 64K THP to ``always``, > > set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M >
