On Thu, 2 Jan 2025 10:22:14 -0500 Gregory Price <gourry@xxxxxxxxxx> wrote: > On Thu, Jan 02, 2025 at 04:09:38AM +0000, Matthew Wilcox wrote: > > On Wed, Jan 01, 2025 at 02:20:39PM -0800, SeongJae Park wrote: > > > Hi all, > > > > > > > > > I find a few interesting and promising projects that aim to do efficient access > > > pattern-aware memory management of near future, including below (alphabetically > > > sorted). > > > > > > - CXL hotness monitoring unit > > > (https://lore.kernel.org/20241121101845.1815660-1-Jonathan.Cameron@xxxxxxxxxx) > > > - Memory tiering fainess by per-cgroup control of promotion and demotion > > > (https://lore.kernel.org/20241108190152.3587484-1-kaiyang2@xxxxxxxxxx) > > > - Promotion of unmapped page cache folios > > > (https://lore.kernel.org/20241210213744.2968-1-gourry@xxxxxxxxxx) > > > > I'm not sure how DAMON can help with this one. As I understand DAMON, > > it monitors accesses to user addresses. This patchset is trying to solve > > the problem for file pages which aren't mapped to userspace at all. > > ie only accessed through read() and write(). > > DAMON can monitor physical addresses to, though the mechanism is > different. Thank you for answering this, Gregory. As Gregory explained, users can use physical address monitoring mode of DAMON for this. For unmapped pages, DAMON sets and reads PG_idle to check if it is accessed or not. Since PG_idle is respected by read() and write() use case to my understanding, DAMON should be able to check accesses to unmapped pages. > I haven't assessed this as a solution, yet. To quickly see this, I ran below simple test. First, I start DAMON in physical address space monitoring mode, wait for one minutes to let it monitor the accesses of the system, and show the access pattern on the system in access temperature histogram format. $ sudo ./damo start $ sleep 60 $ sudo ./damo report access --style temperature-sz-hist <temperature> <total size> [-7,480,000,000, -7,479,999,999) 59.868 GiB | | total size: 59.868 GiB The access temperature histogram format shows size of memory of given access temperature range. Access temperature is a metric that represents the access hotness. If any access to the region is continuously found, the value increases. If no access to the region is found, the temperature becomes zero. If it continues showing no access, the temperature further decreases (goes to minus). Refer to the document[1] for more details. So from the above output, we can show all memory of the system is not accessed at all for the last minute. Now I start a program that continuously overwrites 10 GiB file in background. Attaching the source code (Attachment 0, dd_like.c) at the bottom of this mail. After a few seconds, I show the temperature histogram again. $ sudo ./damo report access --style temperature-sz-hist <temperature> <total size> [-12,590,000,000, -11,699,000,000) 42.038 GiB |********************| [-11,699,000,000, -10,808,000,000) 0 B | | [-10,808,000,000, -9,917,000,000) 0 B | | [-9,917,000,000, -9,026,000,000) 0 B | | [-9,026,000,000, -8,135,000,000) 5.986 GiB |*** | [-8,135,000,000, -7,244,000,000) 0 B | | [-7,244,000,000, -6,353,000,000) 0 B | | [-6,353,000,000, -5,462,000,000) 0 B | | [-5,462,000,000, -4,571,000,000) 0 B | | [-4,571,000,000, -3,680,000,000) 5.951 GiB |*** | [-3,680,000,000, -2,789,000,000) 5.893 GiB |*** | total size: 59.868 GiB We can show DAMON found about 10 GiB relatively hot regions. This is a very simple test that not well tuned. Maybe because of that, there are details to investigate, including why the 10 GiB regions are having different and negative access temperature. I'll skip those for now, since the point of this test is that DAMON at least somehow react to accesses for unmapped pages. [1] https://github.com/damonitor/damo/blob/next/USAGE.md#access-temperature Thanks, SJ > > ~Gregory ==== Attachment 0 (dd_like.c) ==== #include <stdio.h> #include <stdlib.h> #include <string.h> #include <fcntl.h> #include <unistd.h> #include <errno.h> int main(int argc, char *argv[]) { int block_size, count; char *dest_path; if (argc != 4) { printf("Usage: ./dd_simulator <block_size> <count> <destination_file>\n"); return 1; } block_size = atoi(argv[1]); count = atoi(argv[2]); dest_path = argv[3]; // Validate input parameters if (block_size <= 0 || count <= 0) { fprintf(stderr, "Invalid block size or count\n"); return 1; } // Write block size of zeroes 'count' times char *zeroes = calloc(block_size, sizeof(char)); if (!zeroes) { fprintf(stderr, "Memory allocation failed\n"); return 1; } while (1) { // Open destination file in write mode FILE *dest_file = fopen(dest_path, "w"); if (!dest_file) { fprintf(stderr, "Failed to open %s for writing: %s\n", dest_path, strerror(errno)); return 1; } // Write block size of zeroes 'count' times for (int i = 0; i < count; i++) fwrite(zeroes, block_size, 1, dest_file); fclose(dest_file); printf("one pass"); } free(zeroes); return 0; }
