Hi everybody,
We'd like to discuss formalizing a mechanism to induce memory reclaim by
the kernel.
The current multigenerational LRU proposal introduces a debugfs
mechanism[1] for this. The "TMO: Transparent Memory Offloading in
Datacenters" paper also discusses a per-memcg mechanism[2]. While the
former can be used for debugging of MGLRU, both can quite powerfully be
used for proactive reclaim.
Google's datacenters use a similar per-memcg mechanism for the same
purpose. Thus, formalizing the mechanism would allow our userspace to use
an upstream supported interface that will be stable and consistent.
This could be an incremental addition to MGLRU's lru_gen debugfs mechanism
but, since the concept has no direct dependency on the work, we believe it
is useful independent of the reclaim mechanism in use (both with and
without CONFIG_LRU_GEN).
Idea: introduce a per-node sysfs mechanism for inducing memory reclaim
that can be useful for global (non-memcg constrained) reclaim and possible
even if memcg is not enabled in the kernel or mounted. This could
optionally take a memcg id to induce reclaim for a memcg hierarchy.
IOW, this would be a /sys/devices/system/node/nodeN/reclaim mechanim for
each NUMA node N on the system. (It would be similar to the existing
per-node sysfs "compact" mechanism used to trigger compaction from
userspace.)
Userspace would write the following to this file:
- nr_to_reclaim pages
- swappiness factor
- memcg_id of the hierarchy to reclaim from, if any[*]
- flags to specify context, if any[**]
[*] if global reclaim or memcg is not enabled/mounted, this is 0 since
this is the return value of mem_cgroup_id()
[**] this is offered for extensibility to specify the context in which
reclaim is being done (clean file pages only, demotion for memory
tiering vs eviction, etc), otherwise 0
An alternative may be to introduce a /sys/kernel/mm/reclaim mechanism that
also takes a nodemask to reclaim from. The kernel would reclaim memory
over the set of nodes passed to it.
Some questions to get discussion going:
- Overall feedback or suggestions for the proposal in general?
- This proposal uses a value specified in pages to reclaim; this could be
a number of bytes instead. I have no strong opinion, does anybody
else?
- Should this be a per-node mechanism under sysfs like the existing
"compact" mechanism or should it be implemented as a single file that
can optionally specify a nodemask to reclaim from?
Thanks!
[1] https://lore.kernel.org/linux-mm/20220208081902.3550911-12-yuzhao@xxxxxxxxxx
[2] https://dl.acm.org/doi/10.1145/3503222.3507731 (Section 3.3)
