On 09.06.20 11:17, SeongJae Park wrote: > On Tue, 9 Jun 2020 10:47:45 +0200 David Hildenbrand <david@xxxxxxxxxx> wrote: > >> On 09.06.20 08:53, SeongJae Park wrote: >>> From: SeongJae Park <sjpark@xxxxxxxxx> >>> >>> In many cases, users might use DAMON for simple data access aware >>> memory management optimizations such as applying an operation scheme to >>> a memory region of a specific size having a specific access frequency >>> for a specific time. For example, "page out a memory region larger than >>> 100 MiB but having a low access frequency more than 10 minutes", or "Use >>> THP for a memory region larger than 2 MiB having a high access frequency >>> for more than 2 seconds". >>> >>> To minimize users from spending their time for implementation of such >>> simple data access monitoring-based operation schemes, this commit makes >>> DAMON to handle such schemes directly. With this commit, users can >>> simply specify their desired schemes to DAMON. >> >> What would be the alternative? How would a solution where these policies >> are handled by user space (or inside an application?) look like? > > Most simple form of the altermative solution would be doing offline data access > pattern profiling using DAMON and modifying the application source code or > system configuration based on the profiling results. > > More automated alternative solution would be a daemon constructed with two > modules: > > - monitor: monitors the data access pattern of the workload via the DAMON > debugfs interface > - memory manager: based on the monitoring result, make appropriate memory > management changes via mlock(), madvise(), sysctl, etc. > > The daemon would be able to run inside the application process as a thread, or > outside as a standalone process. If the daemon could not run inside the > application process, the memory management changes it could make would be > further limited, though, as mlock() and madvise() would not be available. The > madvise_process(), which is already merged in the next tree, would be helpful > in this case. > >>> >>> Each of the schemes is composed with conditions for filtering of the >>> target memory regions and desired memory management action for the >>> target. Specifically, the format is:: >>> >>> <min/max size> <min/max access frequency> <min/max age> <action> >>> >>> The filtering conditions are size of memory region, number of accesses >>> to the region monitored by DAMON, and the age of the region. The age of >>> region is incremented periodically but reset when its addresses or >>> access frequency has significantly changed or the action of a scheme was >>> applied. For the action, current implementation supports only a few of >>> madvise() hints, ``MADV_WILLNEED``, ``MADV_COLD``, ``MADV_PAGEOUT``, >>> ``MADV_HUGEPAGE``, and ``MADV_NOHUGEPAGE``. >> >> I am missing some important information. Is this specified for *all* >> user space processes? Or how is this configured? What are examples? >> >> E.g., messing with ``MADV_HUGEPAGE`` vs. ``MADV_NOHUGEPAGE`` of random >> applications can change the behavior/break these applications. (e.g., if >> userfaultfd is getting used and the applciation explicitly sets >> MADV_NOHUGEPAGE). > > Only monitoring target processes will be applied. The monitoring target > processes can be specified by writing the process ids to 'pids' debugfs file or > constructing the 'struct damon_ctx' via the programming interface. > > I will refine the commit message to make the points clearer, in the next spin. Understood, so a process configures damon to only modify its mappings. thanks for clarifying! This makes exposing the do_madvise() look less dangerous. -- Thanks, David / dhildenb
