This patch extends a) process_madvise(2) support vector address ranges
in a system call and then b) support the vector address ranges to
local process as well as external process.
Android app has thousands of vmas due to zygote so it's totally waste
of CPU and power if we should call the syscall one by one for each vma.
(With testing 2000-vma syscall vs 1-vector syscall, it showed 15%
performance improvement. I think it would be bigger in real practice
because the testing ran very cache friendly environment).
Another potential use case for the vector range is to amortize the
cost of TLB shootdowns for multiple ranges when using MADV_DONTNEED;
this could benefit users like TCP receive zerocopy and malloc
implementations. In future, we could find more usecases for other
advises so let's make it happens as API since we introduce a new syscall
at this moment. With that, existing madvise(2) user could replace it with
process_madvise(2) with their own pid if they want to have batch address
ranges support feature.
So finally, the API is as follows,
ssize_t process_madvise(idtype_t idtype, id_t id,
const struct iovec *iovec, unsigned long vlen,
int advice, unsigned long flags);
DESCRIPTION
The process_madvise() system call is used to give advice or directions
to the kernel about the address ranges from external process as well as
local process. It provides the advice to address ranges of process
described by iovec and vlen. The goal of such advice is to improve system
or application performance.
The idtype and id arguments select the target process to be advised as
follows:
idtype == P_PID
select the process whose process ID matches id
idtype == P_PIDFD
select the process referred to by the PID file descriptor
specified in id. (See pidofd_open(2) for further information)
The pointer iovec points to an array of iovec structures, defined in
<sys/uio.h> as:
struct iovec {
void *iov_base; /* starting address */
size_t iov_len; /* number of bytes to be advised */
};
The iovec describes address ranges beginning at address(iov_base)
and with size length of bytes(iov_len).
The vlen represents the number of elements in iovec.
The advice is indicated in the advice argument, which is one of the
following at this moment if the target process specified by idtype and
id is external.
MADV_COLD
MADV_PAGEOUT
MADV_MERGEABLE
MADV_UNMERGEABLE
Permission to provide a hint to external process is governed by a
ptrace access mode PTRACE_MODE_ATTACH_FSCREDS check; see ptrace(2).
The process_madvise supports every advice madvise(2) has if target
process is in same thread group with calling process so user could
use process_madvise(2) to extend existing madvise(2) to support
vector address ranges.
RETURN VALUE
On success, process_madvise() returns the number of bytes advised.
This return value may be less than the total number of requested
bytes, if an error occurred. The caller should check return value
to determine whether a partial advice occurred.
Cc: David Rientjes <rientjes@xxxxxxxxxx>
Cc: Arjun Roy <arjunroy@xxxxxxxxxx>
Cc: Tim Murray <timmurray@xxxxxxxxxx>
Cc: Daniel Colascione <dancol@xxxxxxxxxx>
Cc: Sonny Rao <sonnyrao@xxxxxxxxxx>
Cc: Brian Geffon <bgeffon@xxxxxxxxxx>
Cc: Shakeel Butt <shakeelb@xxxxxxxxxx>
Cc: John Dias <joaodias@xxxxxxxxxx>
Cc: Joel Fernandes <joel@xxxxxxxxxxxxxxxxx>
Cc: SeongJae Park <sj38.park@xxxxxxxxx>
Cc: Oleksandr Natalenko <oleksandr@xxxxxxxxxx>
Cc: Suren Baghdasaryan <surenb@xxxxxxxxxx>
Cc: Sandeep Patil <sspatil@xxxxxxxxxx>
Cc: Michal Hocko <mhocko@xxxxxxxx>
Cc: Johannes Weiner <hannes@xxxxxxxxxxx>
Cc: Vlastimil Babka <vbabka@xxxxxxx>
Cc: <linux-man@xxxxxxxxxxxxxxx>
Signed-off-by: Minchan Kim <minchan@xxxxxxxxxx>
---
mm/madvise.c | 47 ++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 40 insertions(+), 7 deletions(-)
diff --git a/mm/madvise.c b/mm/madvise.c
index 097506466fdc..3082d7fa64ee 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -1195,20 +1195,39 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
return do_madvise(current, current->mm, start, len_in, behavior);
}
-SYSCALL_DEFINE6(process_madvise, int, which, pid_t, upid, unsigned long, start,
- size_t, len_in, int, behavior, unsigned long, flags)
+static int do_process_madvise(struct task_struct *target_task,
+ struct mm_struct *mm, struct iov_iter *iter, int behavior)
{
- int ret;
+ struct iovec iovec;
+ int ret = 0;
+
+ while (iov_iter_count(iter)) {
+ iovec = iov_iter_iovec(iter);
+ ret = do_madvise(target_task, mm, (unsigned long)iovec.iov_base,
+ iovec.iov_len, behavior);
+ if (ret < 0)
+ break;
+ iov_iter_advance(iter, iovec.iov_len);
+ }
+
+ return ret;
+}
+
+SYSCALL_DEFINE6(process_madvise, int, which, pid_t, upid,
+ const struct iovec __user *, vec, unsigned long, vlen,
+ int, behavior, unsigned long, flags)
+{
+ ssize_t ret;
struct pid *pid;
struct task_struct *task;
struct mm_struct *mm;
+ struct iovec iovstack[UIO_FASTIOV];
+ struct iovec *iov = iovstack;
+ struct iov_iter iter;
if (flags != 0)
return -EINVAL;
- if (!process_madvise_behavior_valid(behavior))
- return -EINVAL;
-
switch (which) {
case P_PID:
if (upid <= 0)
@@ -1236,13 +1255,27 @@ SYSCALL_DEFINE6(process_madvise, int, which, pid_t, upid, unsigned long, start,
goto put_pid;
}
+ if (task->mm != current->mm &&
+ !process_madvise_behavior_valid(behavior)) {
+ ret = -EINVAL;
+ goto release_task;
+ }
+
mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS);
if (IS_ERR_OR_NULL(mm)) {
ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH;
goto release_task;
}
- ret = do_madvise(task, mm, start, len_in, behavior);
+ ret = import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), &iov, &iter);
+ if (ret >= 0) {
+ size_t total_len = iov_iter_count(&iter);
+
+ ret = do_process_madvise(task, mm, &iter, behavior);
+ if (ret >= 0)
+ ret = total_len - iov_iter_count(&iter);
+ kfree(iov);
+ }
mmput(mm);
release_task:
put_task_struct(task);
--
2.26.1.301.g55bc3eb7cb9-goog
