Yuanchu Xie <yuanchu@xxxxxxxxxx> writes: > Introduce a way of monitoring the working set of a workload, per page > type and per NUMA node, with granularity in minutes. It has page-level > granularity and minimal memory overhead by building on the > Multi-generational LRU framework, which already has most of the > infrastructure and is just missing a useful interface. > > MGLRU organizes pages in generations, where an older generation contains > colder pages, and aging promotes the recently used pages into the young > generation and creates a new one. The working set size is how much > memory an application needs to keep working, the amount of "hot" memory > that's frequently used. The only missing pieces between MGLRU > generations and working set estimation are a consistent aging cadence > and an interface; we introduce the two additions. So with kold kthread do we need aging in reclaim ? Should we switch reciam to wakeup up kold kthread to do aging instead of doing try_to_inc_max_seq? This would also help us to try different aging mechanism which can run better in a kthread. > > Periodic aging > ====== > MGLRU Aging is currently driven by reclaim, so the amount of time > between generations is non-deterministic. With memcgs being aged > regularly, MGLRU generations become time-based working set information. > > - memory.periodic_aging: a new root-level only file in cgroupfs > Writing to memory.periodic_aging sets the aging interval and opts into > periodic aging. > - kold: a new kthread that ages memcgs based on the set aging interval. > > Page idle age stats > ====== > - memory.page_idle_age: we group pages into idle age ranges, and present > the number of pages per node per pagetype in each range. This > aggregates the time information from MGLRU generations hierarchically. > > Use case: proactive reclaimer > ====== > The proactive reclaimer sets the aging interval, and periodically reads > the page idle age stats, forming a working set estimation, which it then > calculates an amount to write to memory.reclaim. > > With the page idle age stats, a proactive reclaimer could calculate a > precise amount of memory to reclaim without continuously probing and > inducing reclaim. > > A proactive reclaimer that uses a similar interface is used in the > Google data centers. > > Use case: workload introspection > ====== > A workload may use the working set estimates to adjust application > behavior as needed, e.g. preemptively killing some of its workers to > avoid its working set thrashing, or dropping caches to fit within a > limit. > It can also be valuable to application developers, who can benefit from > an out-of-the-box overview of the application's usage behaviors. > > TODO List > ====== > - selftests > - a userspace demonstrator combining periodic aging, page idle age > stats, memory.reclaim, and/or PSI > > Open questions > ====== > - MGLRU aging mechanism has a flag called force_scan. With > force_scan=false, invoking MGLRU aging when an lruvec has a maximum > number of generations does not actually perform aging. > However, with force_scan=true, MGLRU moves the pages in the oldest > generation to the second oldest generation. The force_scan=true flag > also disables some optimizations in MGLRU's page table walks. > The current patch sets force_scan=true, so that periodic aging would > work without a proactive reclaimer evicting the oldest generation. > > - The page idle age format uses a fixed set of time ranges in seconds. > I have considered having it be based on the aging interval, or just > compiling the raw timestamps. > With the age ranges based on the aging interval, a memcg that's > undergoing memcg reclaim might have its generations in the 10 > seconds range, and a much longer aging interval would obscure this > fact. > The raw timestamps from MGLRU could lead to a very large file when > aggregated hierarchically. > > Yuanchu Xie (2): > mm: multi-gen LRU: periodic aging > mm: multi-gen LRU: cgroup working set stats > > include/linux/kold.h | 44 ++++++++++ > include/linux/mmzone.h | 4 +- > mm/Makefile | 3 + > mm/kold.c | 150 ++++++++++++++++++++++++++++++++ > mm/memcontrol.c | 188 +++++++++++++++++++++++++++++++++++++++++ > mm/vmscan.c | 35 +++++++- > 6 files changed, 422 insertions(+), 2 deletions(-) > create mode 100644 include/linux/kold.h > create mode 100644 mm/kold.c > > -- > 2.39.0.314.g84b9a713c41-goog
