On Mon 21-03-22 08:46:35, Zach O'Keefe wrote: > Hey Michal, thanks for taking the time to review / comment. > > On Mon, Mar 21, 2022 at 7:38 AM Michal Hocko <mhocko@xxxxxxxx> wrote: > > > > [ Removed Richard Henderson from the CC list as the delivery fails for > > his address] > > Thank you :) > > > On Tue 08-03-22 13:34:03, Zach O'Keefe wrote: > > > Introduction > > > -------------------------------- > > > > > > This series provides a mechanism for userspace to induce a collapse of > > > eligible ranges of memory into transparent hugepages in process context, > > > thus permitting users to more tightly control their own hugepage > > > utilization policy at their own expense. > > > > > > This idea was previously introduced by David Rientjes, and thanks to > > > everyone for your patience while I prepared these patches resulting from > > > that discussion[1]. > > > > > > [1] https://lore.kernel.org/all/C8C89F13-3F04-456B-BA76-DE2C378D30BF@xxxxxxxxxx/ > > > > > > Interface > > > -------------------------------- > > > > > > The proposed interface adds a new madvise(2) mode, MADV_COLLAPSE, and > > > leverages the new process_madvise(2) call. > > > > > > (*) process_madvise(2) > > > > > > Performs a synchronous collapse of the native pages mapped by > > > the list of iovecs into transparent hugepages. The default gfp > > > flags used will be the same as those used at-fault for the VMA > > > region(s) covered. > > > > Could you expand on reasoning here? The default allocation mode for #PF > > is rather light. Madvised will try harder. The reasoning is that we want > > to make stalls due to #PF as small as possible and only try harder for > > madvised areas (also a subject of configuration). Wouldn't it make more > > sense to try harder for an explicit calls like madvise? > > > > The reasoning is that the user has presumably configured system/vmas > to tell the kernel how badly they want thps, and so this call aligns > with current expectations. I.e. a user who goes about the trouble of > trying to fault-in a thp at a given memory address likely wants a thp > "as bad" as the same user MADV_COLLAPSE'ing the same memory to get a > thp. If the syscall tries only as hard as the #PF doesn't that limit the functionality? I mean a non #PF can consume more resources to allocate and collapse a THP as it won't inflict any measurable latency to the targetting process (except for potential CPU contention). From that perspective madvise is much more similar to khugepaged. I would even argue that it could try even harder because madvise is focused on a very specific memory range and the execution is not shared among all processes that are scanned by khugepaged. > If this is not the case, then the MADV_F_COLLAPSE_DEFRAG flag could be > used to explicitly request the kernel to try harder, as you mention. Do we really need that? How many do_harder levels do we want to support? What would be typical usecases for #PF based and DEFRAG usages? [...] > > > Diverging from the at-fault semantics, VM_NOHUGEPAGE is ignored > > > by default, as the user is explicitly requesting this action. > > > Define two flags to control collapse semantics, passed through > > > process_madvise(2)’s optional flags parameter: > > > > This part is discussed later in the thread. > > > > > > > > MADV_F_COLLAPSE_LIMITS > > > > > > If supplied, collapse respects pte collapse limits set via > > > sysfs: > > > /transparent_hugepage/khugepaged/max_ptes_[none|swap|shared]. > > > Required if calling on behalf of another process and not > > > CAP_SYS_ADMIN. > > > > > > MADV_F_COLLAPSE_DEFRAG > > > > > > If supplied, permit synchronous compaction and reclaim, > > > regardless of VMA flags. > > > > Why do we need this? > > Do you mean MADV_F_COLLAPSE_DEFRAG specifically, or both? > > * MADV_F_COLLAPSE_LIMITS is included because we'd like some form of > inter-process protection for collapsing memory in another process' > address space (which a malevolent program could exploit to cause oom > conditions in another memcg hierarchy, for example), but we want > privileged (CAP_SYS_ADMIN) users to otherwise be able to optimize thp > utilization as they wish. Could you expand some more please? How is this any different from khugepaged (well, except that you can trigger the collapsing explicitly rather than rely on khugepaged to find that mm)? > * MADV_F_COLLAPSE_DEFRAG is useful as mentioned above, where we want > to explicitly tell the kernel to try harder to back this by thps, > regardless of the current system/vma configuration. > > Note that when used together, these flags can be used to implement the > exact behavior of khugepaged, through MADV_COLLAPSE. IMHO this is stretching the interface and this can backfire in the future. The interface should be really trivial. I want to collapse a memory area. Let the kernel do the right thing and do not bother with all the implementation details. I would use the same allocation strategy as khugepaged as this seems to be closesest from the latency and application awareness POV. In a way you can look at the madvise call as a way to trigger khugepaged functionality on he particular memory range. -- Michal Hocko SUSE Labs
