On 12/28/2021 4:44 PM, SeongJae Park wrote:
Hello,
On Mon, 27 Dec 2021 11:09:56 +0800 "Huang, Ying" <ying.huang@xxxxxxxxx> wrote:
Hi, SeongJae,
SeongJae Park <sj@xxxxxxxxxx> writes:
Hi,
On Thu, 23 Dec 2021 15:51:18 +0800 "Huang, Ying" <ying.huang@xxxxxxxxx> wrote:
[snip]
It's good to avoid to change the source code of an application to apply
some memory management optimization (for example, use DAMON +
madvise()). But it's much easier to run a user space daemon to optimize
for the application. (for example, use DAMON + other information +
process_madvise()).
And this kind of per-application optimization is kind of application
specific policy. This kind of policy may be too complex and flexible to
be put in the kernel directly. For example, in addition to DAMON, some
other application specific or system knowledge may be helpful too, so we
have process_madvise() for that before DAMON. Some more complex
algorithm may be needed for some applications.
And this kind of application specific policy usually need complex
configuration. It's hard to export all these policy parameters to the
user space as the kernel ABI. Now, DAMON schemes parameters are
exported in debugfs so they are not considered ABI. So they may be
changed at any time. But applications need some stable and
well-maintained ABI.
All in all, IMHO, what we need is a user space per-application policy
daemon with the information from DAMON and other sources.
I basically agree to Ying, as I also noted in the coverletter of DAMOS
patchset[1]:
DAMON[1] can be used as a primitive for data access aware memory
management optimizations. For that, users who want such optimizations
should run DAMON, read the monitoring results, analyze it, plan a new
memory management scheme, and apply the new scheme by themselves. Such
efforts will be inevitable for some complicated optimizations.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=fda504fade7f124858d7022341dc46ff35b45274
That is, I believe some programs and big companies would definitely have their
own information and want such kind of complicated optimizations. But, such
optimizations would depend on characteristics of each program and require
investment of some amount of resources. Some other programs and users wouldn't
have such special information, and/or resource to invest for such
optimizations. For them, some amount of benefit would be helpful enough even
though its sub-optimal.
I think we should help both groups, and DAMOS could be useful for the second
group. And I don't think DAMOS is useless for the first group. They could use
their information-based policy in prallel to DAMOS in some cases. E.g., if
they have a way to predict the data access pattern of specific memory region
even without help from DAMON, they can use their own policy for the region but
DAMOS for other regions.
Someone could ask why not implement a user-space implementation for the second
group, then. First of all, DAMOS is not only for the user-space driven virtual
memory management optimization, but also for kernel-space programs and any
DAMOS-supportable address spaces including the physical address space. And,
another important goal of DAMOS for user space driven use case in addition to
reducing the redundant code is minimizing the user-kernel context switch
overhead for passing the monitoring results information and memory management
action requests.
In summary, I agree the user space per-application policy daemon will be useful
for the specialized ultimate optimizations, but we also need DAMOS for another
common group of cases.
If I'm missing something, please feel free to let me know.
I guess that most end-users and quite some system administrators of
small companies have no enough capability to take advantage of the
per-application optimizations. How do they know the appropriate region
number and proactive reclaim threshold?
So per my understanding, Linux kernel
need provide,
1. An in-kernel general policy that is obviously correct and benefits
almost all users and applications, at least no regression. No
complex configuration or deep knowledge is needed to take advantage
of it.
2. Some way to inspect and control system and application behavior, so
that some advanced and customized user space policy daemons can be
built to satisfy some advanced users who have the enough knowledge
for the applications and systems, for example, oomd.
Agreed, and I think that's the approach that DAMON is currently taking. In
specific, we provide DAMON debugfs interface for users who want to inspect and
control their system and application behavior. Using it, we also made a PoC
level user space policy daemon[1].
For the in-kernel policies, we are developing DAMON-based kernel components one
by one, for specific usages. DAMON-based proactive reclamation module
(DAMON_RECLAIM) is one such example. Such DAMON-based components will remove
complex tunables that necessary for the general inspection and control of the
system but unnecessary for their specific purpose (e.g., proactive reclamation)
to allow users use it in a simple manner. Also, those will use conservative
default configs to not incur visible regression. For example, DAMON_RECLAIM
uses only up to 1% of single CPU time for the reclamation by default.
In short, I think we're on the same page, and adding DEMOTION scheme action
could be helpful for the users who want to efficiently inspect and control the
system/application behavior for their tiered memory systems. It's unclear how
Agree. It will be easier for us to deploy it to the products for the
common scenarios.
much benefit this could give to users, though. I assume Baolin would come back
with some sort of numbers in the next spin. Nevertheless, I personally don't
Yes, I am still trying to set up the effective measurement environment
and get the performance number in the next version.
think that's a critical blocker, as this patch is essentially just adding a way
for using the pre-existing primitive, namely move_pages(), in a little bit more
efficient manner, for the access pattern-based use cases.
If I'm missing something, please feel free to let me know.
[1] https://github.com/awslabs/damoos
