On Sun, Mar 6, 2022 at 4:11 PM David Rientjes <rientjes@xxxxxxxxxx> wrote: > > Hi everybody, > > We'd like to discuss formalizing a mechanism to induce memory reclaim by > the kernel. > > The current multigenerational LRU proposal introduces a debugfs > mechanism[1] for this. The "TMO: Transparent Memory Offloading in > Datacenters" paper also discusses a per-memcg mechanism[2]. While the > former can be used for debugging of MGLRU, both can quite powerfully be > used for proactive reclaim. > > Google's datacenters use a similar per-memcg mechanism for the same > purpose. Thus, formalizing the mechanism would allow our userspace to use > an upstream supported interface that will be stable and consistent. > > This could be an incremental addition to MGLRU's lru_gen debugfs mechanism > but, since the concept has no direct dependency on the work, we believe it > is useful independent of the reclaim mechanism in use (both with and > without CONFIG_LRU_GEN). > > Idea: introduce a per-node sysfs mechanism for inducing memory reclaim > that can be useful for global (non-memcg constrained) reclaim and possible > even if memcg is not enabled in the kernel or mounted. This could > optionally take a memcg id to induce reclaim for a memcg hierarchy. > > IOW, this would be a /sys/devices/system/node/nodeN/reclaim mechanim for > each NUMA node N on the system. (It would be similar to the existing > per-node sysfs "compact" mechanism used to trigger compaction from > userspace.) > > Userspace would write the following to this file: > - nr_to_reclaim pages > - swappiness factor > - memcg_id of the hierarchy to reclaim from, if any[*] > - flags to specify context, if any[**] > > [*] if global reclaim or memcg is not enabled/mounted, this is 0 since > this is the return value of mem_cgroup_id() > [**] this is offered for extensibility to specify the context in which > reclaim is being done (clean file pages only, demotion for memory > tiering vs eviction, etc), otherwise 0 > > An alternative may be to introduce a /sys/kernel/mm/reclaim mechanism that > also takes a nodemask to reclaim from. The kernel would reclaim memory > over the set of nodes passed to it. > > Some questions to get discussion going: > > - Overall feedback or suggestions for the proposal in general? > > - This proposal uses a value specified in pages to reclaim; this could be > a number of bytes instead. I have no strong opinion, does anybody > else? > > - Should this be a per-node mechanism under sysfs like the existing > "compact" mechanism or should it be implemented as a single file that > can optionally specify a nodemask to reclaim from? > > Thanks! > > [1] https://lore.kernel.org/linux-mm/20220208081902.3550911-12-yuzhao@xxxxxxxxxx > [2] https://dl.acm.org/doi/10.1145/3503222.3507731 (Section 3.3) Adding Canonical who also provided additional use cases [3] for this potential ABI. [3] https://lore.kernel.org/lkml/20201005081313.732745-1-andrea.righi@xxxxxxxxxxxxx/
