From: SeongJae Park <sjpark@xxxxxxxxx> DAMON and DAMOS --------------- DAMON[1] is a framework for general data access monitoring of kernel subsystems. It provides best-effort high quality monitoring results while incurring only minimal and upper-bounded overhead, due to its practical overhead-accuracy tradeoff mechanism. On a production machine utilizing 70 GB memory, it can repeatedly scan accesses to the whole memory for every 5ms, while consuming only 1% single CPU time. On top of it, a data access pattern-oriented memory management engine called DAMON-based Operation Schemes (DAMOS) is implemented. It allows clients to implement their access pattern oriented memory management logic with very simple scheme descriptions. We implemented fine-grained access-aware THP and proactive reclamation using this engine in three lines of scheme and achieved remarkable improvements[2]. As of this writing (2021-05-28), the code is not in the mainline but available at its development tree[3], and regularly posted to LKML as patchsets[4,5,6]. Nevertheless, the code has already merged in the public Amazon Linux kernel trees[7,8], and all Amazon Linux users can use DAMON/DAMOS off the box. We are also supporting the two latest upstream LTS stable kernels[9,10]. Agenda ------ Now we can improve Linux kernel's memory management using DAMON's fine-grained access information. We could do that by modifying existing subsystems to use DAMON internally, or implementing new ones on top of DAMOS and let users select. The two ways would have their own pros and cons. I am currently working for implementation of another practical and lightweight proactive reclamation in the latter way, because this is a new subsystem. I'd like to further extend this work to other parts of the kernel including THP, NUMA and tiered-memory balancing, LRU list arrangement, and KVM, though I don't have concrete idea as of now but only shallow thought. I'd like to briefly introduce DAMON/DAMOS, my progress and plans, and discuss with people for - thoughts on the programming interface of DAMON/DAMOS, - pros and cons of the two approaches for this kind of work, - how DAMON/DAMOS could collaborate with other similar works, - concerns regarding my planned future works, and - what additional DAMON/DAMOS-based optimizations would be possible. [1] https://damonitor.github.io (https://damonitor.github.io/) [2] https://damonitor.github.io/doc/html/latest/vm/damon/eval.html [3] https://https://github.com/sjp38/linux/tree/damon/master (https://https//github.com/sjp38/linux/tree/damon/master) [4] https://lore.kernel.org/linux-mm/20210520075629.4332-1-sj38.park@xxxxxxxxx/ [5] https://lore.kernel.org/linux-mm/20201216084404.23183-1-sjpark@xxxxxxxxxx/ [6] https://lore.kernel.org/linux-mm/20201216094221.11898-1-sjpark@xxxxxxxxxx/ [7] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon [8] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damon [9] https://github.com/sjp38/linux/tree/damon/for-v5.4.y [10] https://github.com/sjp38/linux/tree/damon/for-v5.10.y
