There were a number of previous attempts to upstream support for anonymous
VMA naming. The original submission by Colin Cross [1] implemented a
dictionary of refcounted names to reuse same name strings. Dave Hansen
suggested [2] to use userspace pointers instead and the patch was rewritten
that way. The last v7 version of this patch was posted by Sumit Semwal [3]
and a very similar patch has been used in Android to name anonymous VMAs
for a number of years. Concerns about this patch were raised by Kees Cook
[4] noting the lack of string sanitization and the use of userspace
pointers from the kernel. In conclusion [5], it was suggested to
strndup_user the strings from userspace, perform appropriate checks and
store a copy as a vm_area_struct member. Performance impact from
additional strdup's during fork() should be measured by allocating a large
number (64k) of VMAs with longest names and timing fork()s.
This patchset implements the suggested approach in the first 2 patches and
the 3rd patch implements simple refcounting to avoid strdup'ing the names
during fork() and minimize the regression.
Proposed test was conducted on an ARM64 Android device with CPU frequency
locked at 2.4GHz, performance governor and Android system being stopped
(adb shell stop) to minimize the noise. Test includes 3 different
scenarios. In each scenario a process with 64K named anonymous VMAs forks
children 1000 times while timing each fork and reporting the average time.
The scenarios differ in the VMA content:
1. VMAs are not populated with any data (not realistic scenario but
helps in emphasizing the regression).
2. Each VMA contains 1 page populated with random data.
3. Each VMA contains 10 pages populated with random data.
With the first 2 patches implementing strdup approach, the average fork()
times are:
unnamed VMAs named VMAs REGRESSION
Unpopulated VMAs 16.73ms 23.34ms 39.51%
VMAs with 1 page of data 51.98ms 59.94ms 15.31%
VMAs with 10 pages of data 66.86ms 76.31ms 14.13%
>From the perf results, the regression can be attributed to strlen() and
strdup() calls. The regression shrinking with the increased amount of
populated data can be attributed mostly to anon_vma_fork() and
copy_page_range() consuming more time during fork().
After the refcounting implemented in the last patch of this series the
results are:
unnamed VMAs named VMAs REGRESSION
Unpopulated VMAs 16.36ms 18.35ms 12.16%%
VMAs with 1 page of data 48.16ms 51.30ms 6.52%
VMAs with 10 pages of data 64.23ms 67.69ms 5.39%
>From the perf results, the regression can be attributed to
refcount_inc_checked() (called from kref_get()).
While there is obviously a measurable regression, 64K named anonymous VMAs
is truly a worst case scenario. In the real usage, the only current user of
this feature, namely Android, rarely has processes with the number of VMAs
reaching 4000 (that's the highest I've measured). The regression of forking
a process with that number of VMAs is at the noise level.
1. https://lore.kernel.org/linux-mm/1372901537-31033-1-git-send-email-ccross@xxxxxxxxxxx/
2. https://lore.kernel.org/linux-mm/51DDFA02.9040707@xxxxxxxxx/
3. https://lore.kernel.org/linux-mm/20200901161459.11772-1-sumit.semwal@xxxxxxxxxx/
4. https://lore.kernel.org/linux-mm/202009031031.D32EF57ED@keescook/
5. https://lore.kernel.org/linux-mm/5d0358ab-8c47-2f5f-8e43-23b89d6a8e95@xxxxxxxxx/
Colin Cross (2):
mm: rearrange madvise code to allow for reuse
mm: add a field to store names for private anonymous memory
Suren Baghdasaryan (1):
mm: add anonymous vma name refcounting
Documentation/filesystems/proc.rst | 2 +
fs/proc/task_mmu.c | 14 +-
fs/userfaultfd.c | 7 +-
include/linux/mm.h | 13 +-
include/linux/mm_types.h | 55 +++-
include/uapi/linux/prctl.h | 3 +
kernel/fork.c | 2 +
kernel/sys.c | 48 ++++
mm/madvise.c | 447 +++++++++++++++++++----------
mm/mempolicy.c | 3 +-
mm/mlock.c | 2 +-
mm/mmap.c | 38 +--
mm/mprotect.c | 2 +-
13 files changed, 462 insertions(+), 174 deletions(-)
--
2.33.0.259.gc128427fd7-goog
