Hi, This is an attempt towards having a single subsystem that accumulates hot page information from lower memory tiers and does hot page promotion. At the heart of this subsystem is a kernel daemon named kpromoted that does the following: 1. Exposes an API that other subsystems which detect/generate memory access information can use to inform the daemon about memory accesses from lower memory tiers. 2. Maintains the list of hot pages and attempts to promote them to toptiers. Currently I have added AMD IBS driver as one source that provides page access information as an example. This driver feeds info to kpromoted in this RFC patchset. More sources were discussed in a similar context here at [1]. This is just an early attempt to check what it takes to maintain a single source of page hotness info and also separate hot page detection mechanisms from the promotion mechanism. There are too many open ends right now and I have listed a few of them below. - The API that is provided to register memory access expects the PFN, NID and time of access at the minimum. This is described more in patch 2/4. This API currently can be called only from contexts that allow sleeping and hence this rules out using it from PTE scanning paths. The API needs to be more flexible with respect to this. - Some sources like PTE A bit scanning can't provide the precise time of access or the NID that is accessing the page. The latter has been an open problem to which I haven't come across a good and acceptable solution. - The way the hot page information is maintained is pretty primitive right now. Ideally we would like to store hotness info in such a way that it should be easily possible to lookup say N most hot pages. - If PTE A bit scanners are considered as hotness sources, we will be bombarded with accesses. Do we want to accomodate all those accesses or just go with hotness info for fixed number of pages (possibly as a ratio of lower tier memory capacity)? - Undoubtedly the mechanism to classify a page as hot and subsequent promotion needs to be more sophisticated than what I have right now. This is just an early RFC posted now to ignite some discussion in the context of LSFMM [2]. I am also working with Raghu to integrate his kmmdscan [3] as the hotness source and use kpromoted for migration. Also, I had posted the IBS driver ealier as an alternative to hint faults based NUMA Balancing [4]. However here I am using it as generic page hotness source. [1] https://lore.kernel.org/linux-mm/de31971e-98fc-4baf-8f4f-09d153902e2e@xxxxxxx/ [2] https://lore.kernel.org/linux-mm/20250123105721.424117-1-raghavendra.kt@xxxxxxx/ [3] https://lore.kernel.org/all/20241201153818.2633616-1-raghavendra.kt@xxxxxxx/ [3] https://lore.kernel.org/lkml/20230208073533.715-2-bharata@xxxxxxx/ Regards, Bharata. Bharata B Rao (4): mm: migrate: Allow misplaced migration without VMA too mm: kpromoted: Hot page info collection and promotion daemon x86: ibs: In-kernel IBS driver for memory access profiling x86: ibs: Enable IBS profiling for memory accesses arch/x86/events/amd/ibs.c | 11 + arch/x86/include/asm/entry-common.h | 3 + arch/x86/include/asm/hardirq.h | 2 + arch/x86/include/asm/ibs.h | 9 + arch/x86/include/asm/msr-index.h | 16 ++ arch/x86/mm/Makefile | 3 +- arch/x86/mm/ibs.c | 344 ++++++++++++++++++++++++++++ include/linux/kpromoted.h | 54 +++++ include/linux/mmzone.h | 4 + include/linux/vm_event_item.h | 30 +++ mm/Kconfig | 7 + mm/Makefile | 1 + mm/kpromoted.c | 305 ++++++++++++++++++++++++ mm/migrate.c | 5 +- mm/mm_init.c | 10 + mm/vmstat.c | 30 +++ 16 files changed, 831 insertions(+), 3 deletions(-) create mode 100644 arch/x86/include/asm/ibs.h create mode 100644 arch/x86/mm/ibs.c create mode 100644 include/linux/kpromoted.h create mode 100644 mm/kpromoted.c -- 2.34.1
