The patch titled
Subject: dcache: account external names as indirectly reclaimable memory
has been added to the -mm tree. Its filename is
dcache-account-external-names-as-indirectly-reclaimable-memory.patch
This patch should soon appear at
http://ozlabs.org/~akpm/mmots/broken-out/dcache-account-external-names-as-indirectly-reclaimable-memory.patch
and later at
http://ozlabs.org/~akpm/mmotm/broken-out/dcache-account-external-names-as-indirectly-reclaimable-memory.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Roman Gushchin <guro@xxxxxx>
Subject: dcache: account external names as indirectly reclaimable memory
I received a report about suspicious growth of unreclaimable slabs on some
machines. I've found that it happens on machines with low memory
pressure, and these unreclaimable slabs are external names attached to
dentries.
External names are allocated using generic kmalloc() function, so they are
accounted as unreclaimable. But they are held by dentries, which are
reclaimable, and they will be reclaimed under the memory pressure.
In particular, this breaks MemAvailable calculation, as it doesn't take
unreclaimable slabs into account. This leads to a silly situation, when a
machine is almost idle, has no memory pressure and therefore has a big
dentry cache. And the resulting MemAvailable is too low to start a new
workload.
To address the issue, the NR_INDIRECTLY_RECLAIMABLE_BYTES counter is used
to track the amount of memory, consumed by external names. The counter is
increased in the dentry allocation path, if an external name structure is
allocated; and it's decreased in the dentry freeing path.
To reproduce the problem I've used the following Python script:
import os
for iter in range (0, 10000000):
try:
name = ("/some_long_name_%d" % iter) + "_" * 220
os.stat(name)
except Exception:
pass
Without this patch:
$ cat /proc/meminfo | grep MemAvailable
MemAvailable: 7811688 kB
$ python indirect.py
$ cat /proc/meminfo | grep MemAvailable
MemAvailable: 2753052 kB
With the patch:
$ cat /proc/meminfo | grep MemAvailable
MemAvailable: 7809516 kB
$ python indirect.py
$ cat /proc/meminfo | grep MemAvailable
MemAvailable: 7749144 kB
Link: http://lkml.kernel.org/r/20180305133743.12746-5-guro@xxxxxx
Signed-off-by: Roman Gushchin <guro@xxxxxx>
Reviewed-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Alexander Viro <viro@xxxxxxxxxxxxxxxxxx>
Cc: Michal Hocko <mhocko@xxxxxxxx>
Cc: Johannes Weiner <hannes@xxxxxxxxxxx>
Cc: Mel Gorman <mgorman@xxxxxxxxxxxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
fs/dcache.c | 29 ++++++++++++++++++++++++-----
1 file changed, 24 insertions(+), 5 deletions(-)
diff -puN fs/dcache.c~dcache-account-external-names-as-indirectly-reclaimable-memory fs/dcache.c
--- a/fs/dcache.c~dcache-account-external-names-as-indirectly-reclaimable-memory
+++ a/fs/dcache.c
@@ -269,8 +269,16 @@ static void __d_free(struct rcu_head *he
static void __d_free_external(struct rcu_head *head)
{
struct dentry *dentry = container_of(head, struct dentry, d_u.d_rcu);
- kfree(external_name(dentry));
- kmem_cache_free(dentry_cache, dentry);
+ struct external_name *name = external_name(dentry);
+ unsigned long bytes;
+
+ bytes = dentry->d_name.len + offsetof(struct external_name, name[1]);
+ mod_node_page_state(page_pgdat(virt_to_page(name)),
+ NR_INDIRECTLY_RECLAIMABLE_BYTES,
+ -kmalloc_size(kmalloc_index(bytes)));
+
+ kfree(name);
+ kmem_cache_free(dentry_cache, dentry);
}
static inline int dname_external(const struct dentry *dentry)
@@ -1602,6 +1610,7 @@ struct dentry *__d_alloc(struct super_bl
struct dentry *dentry;
char *dname;
int err;
+ size_t reclaimable = 0;
dentry = kmem_cache_alloc(dentry_cache, GFP_KERNEL);
if (!dentry)
@@ -1618,9 +1627,11 @@ struct dentry *__d_alloc(struct super_bl
name = &slash_name;
dname = dentry->d_iname;
} else if (name->len > DNAME_INLINE_LEN-1) {
- size_t size = offsetof(struct external_name, name[1]);
- struct external_name *p = kmalloc(size + name->len,
- GFP_KERNEL_ACCOUNT);
+ struct external_name *p;
+
+ reclaimable = offsetof(struct external_name, name[1]) +
+ name->len;
+ p = kmalloc(reclaimable, GFP_KERNEL_ACCOUNT);
if (!p) {
kmem_cache_free(dentry_cache, dentry);
return NULL;
@@ -1665,6 +1676,14 @@ struct dentry *__d_alloc(struct super_bl
}
}
+ if (unlikely(reclaimable)) {
+ pg_data_t *pgdat;
+
+ pgdat = page_pgdat(virt_to_page(external_name(dentry)));
+ mod_node_page_state(pgdat, NR_INDIRECTLY_RECLAIMABLE_BYTES,
+ kmalloc_size(kmalloc_index(reclaimable)));
+ }
+
this_cpu_inc(nr_dentry);
return dentry;
_
Patches currently in -mm which might be from guro@xxxxxx are
mm-introduce-nr_indirectly_reclaimable_bytes.patch
mm-treat-indirectly-reclaimable-memory-as-available-in-memavailable.patch
dcache-account-external-names-as-indirectly-reclaimable-memory.patch
mm-oom-refactor-the-oom_kill_process-function.patch
mm-implement-mem_cgroup_scan_tasks-for-the-root-memory-cgroup.patch
mm-oom-cgroup-aware-oom-killer.patch
mm-oom-introduce-memoryoom_group.patch
mm-oom-introduce-memoryoom_group-fix.patch
mm-oom-add-cgroup-v2-mount-option-for-cgroup-aware-oom-killer.patch
mm-oom-docs-describe-the-cgroup-aware-oom-killer.patch
mm-oom-docs-describe-the-cgroup-aware-oom-killer-fix.patch
cgroup-list-groupoom-in-cgroup-features.patch
--
To unsubscribe from this list: send the line "unsubscribe mm-commits" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
