Roman's previously posted memory.low,min patches add per memcg effective
low limit to detect overcommitment of parental limits. But if we flip
low,min reclaim to bail if usage<{low,min} at any level, then we don't need
an effective low limit, which makes the code simpler. When parent limits
are overcommited memory.min will oom kill, which is more drastic but makes
the memory.low a simpler concept. If memcg a/b wants oom kill before
reclaim, then give it to them. It seems a bit strange for a/b/memory.low's
behaviour to depend on a/c/memory.low (i.e. a/b.low is strong unless
a/b.low+a/c.low exceed a.low).
Previous patches:
- mm: rename page_counter's count/limit into usage/max
- mm: memory.low hierarchical behavior
- mm: treat memory.low value inclusive
- mm/docs: describe memory.low refinements
- mm: introduce memory.min
- mm: move the high field from struct mem_cgroup to page_counter
8 files changed, 405 insertions(+), 141 deletions(-)
I think there might be a simpler way (ableit it doesn't yet include
Documentation):
- memcg: fix memory.low
- memcg: add memory.min
3 files changed, 75 insertions(+), 6 deletions(-)
The idea of this alternate approach is for memory.low,min to avoid reclaim
if any portion of under-consideration memcg ancestry is under respective
limit.
Also, in either approach, I suspect we should mention the interaction with
numa contrainted allocations (cpuset.mems, mempolicy, mbind). For example,
if a numa agnostic memcg with large memory.min happens to gobble up all of
node N1 memory, and a future task really wants node N1 memory (via
mempolicy) then we oom kill rather reclaim or migrating memory.
Ideas:
a) oom kill numa constrainted allocator, that's what we've been doing in
Google. I can provide patch if helpful. I admit that it has
shortcomings.
b) oom kill a memcg with memory.low protection if its TBD priority is lower
than the allocating task. Priority is a TBD concept.
c) consider migrating numa agnostic memory.low memory as a lighterweight
alternative to oom kill.
I extended Roman's nifty reclaim test:
#!/bin/bash
#
# Uppercase cgroups can tolerate some reclaim (current > low).
# Lowerase cgroups are intolerate to reclaim (current < low).
#
# A A/memory.low = 2G, A/memory.current = 6G
# // \\
# bC De b/memory.low = 3G b/memory.current = 2G
# C/memory.low = 1G C/memory.current = 2G
# D/memory.low = 0 D/memory.current = 2G
# e/memory.low = 10G e/memory.current = 0
#
# F F/memory.low = 2G, F/memory.current = 4G
# / \
# g H g/memory.low = 3G g/memory.current = 2G
# H/memory.low = 1G H/memory.current = 2G
#
# i i/memory.low = 5G, i/memory.current = 4G
# / \
# j K j/memory.low = 3G j/memory.current = 2G
# K/memory.low = 1G K/memory.current = 2G
#
# L L/memory.low = 2G, L/memory.current = 4G, L/memory.max = 4G
# / \
# m N m/memory.low = 3G m/memory.current = 2G
# N/memory.low = 1G N/memory.current = 2G
#
# setting memory.min: warmup => after global pressure
# A : 6306372k => 3154336k
# A/b : 2102184k => 2101928k
# A/C : 2101936k => 1048352k
# A/D : 2102252k => 4056k
# A/e : 0k => 0k
# F : 4204420k => 3150272k
# F/g : 2102188k => 2101912k
# F/H : 2102232k => 1048360k
# i : 4204652k => 4203884k
# i/j : 2102324k => 2101940k
# i/K : 2102328k => 2101944k
# L : 4189976k => 3147824k
# L/m : 2101980k => 2101956k
# L/N : 2087996k => 1045868k
#
# setting memory.min: warmup => after L/m antagonist
# A : 6306076k => 6305988k
# A/b : 2102152k => 2102128k
# A/C : 2101948k => 2101916k
# A/D : 2101976k => 2101944k
# A/e : 0k => 0k
# F : 4204156k => 4203832k
# F/g : 2102220k => 2101920k
# F/H : 2101936k => 2101912k
# i : 4204204k => 4203852k
# i/j : 2102256k => 2101936k
# i/K : 2101948k => 2101916k
# L : 4190012k => 3886352k
# L/m : 2101996k => 2837856k
# L/N : 2088016k => 1048496k
#
# setting memory.low: warmup => after global pressure
# A : 6306220k => 3154864k
# A/b : 2101964k => 2101940k
# A/C : 2102204k => 1047040k
# A/D : 2102052k => 5884k
# A/e : 0k => 0k
# F : 4204192k => 3147888k
# F/g : 2101948k => 2101916k
# F/H : 2102244k => 1045972k
# i : 4204480k => 4204056k
# i/j : 2102008k => 2101976k
# i/K : 2102464k => 2102080k
# L : 4190028k => 3150192k
# L/m : 2102004k => 2101980k
# L/N : 2088024k => 1048212k
#
# setting memory.low: warmup => after L/m antagonist
# A : 6306360k => 6305960k
# A/b : 2101988k => 2101924k
# A/C : 2102192k => 2101916k
# A/D : 2102180k => 2102120k
# A/e : 0k => 0k
# F : 4203964k => 4203908k
# F/g : 2102016k => 2101992k
# F/H : 2101948k => 2101916k
# i : 4204408k => 4203988k
# i/j : 2101984k => 2101936k
# i/K : 2102424k => 2102052k
# L : 4189960k => 3886296k
# L/m : 2101968k => 2838704k
# L/N : 2087992k => 1047592k
set -o errexit
set -o nounset
set -o pipefail
LIM="$1"; shift
ANTAGONIST="$1"; shift
CGPATH=/tmp/cgroup
vmtouch2() {
rm -f "$2"
(echo $BASHPID > "${CGPATH}/$1/cgroup.procs" && exec /tmp/mmap --loop 1 --file "$2" "$3")
}
vmtouch() {
# twice to ensure slab caches are warmed up and all objs are charged to cgroup.
vmtouch2 "$1" "$2" "$3"
vmtouch2 "$1" "$2" "$3"
}
dump() {
for i in A A/b A/C A/D A/e F F/g F/H i i/j i/K L L/m L/N; do
printf "%-5s: %sk\n" $i $(($(cat "${CGPATH}/${i}/memory.current")/1024))
done
}
rm -f /file_?
if [[ -e "${CGPATH}/A" ]]; then
rmdir ${CGPATH}/?/? ${CGPATH}/?
fi
echo "+memory" > "${CGPATH}/cgroup.subtree_control"
mkdir "${CGPATH}/A" "${CGPATH}/F" "${CGPATH}/i" "${CGPATH}/L"
echo "+memory" > "${CGPATH}/A/cgroup.subtree_control"
echo "+memory" > "${CGPATH}/F/cgroup.subtree_control"
echo "+memory" > "${CGPATH}/i/cgroup.subtree_control"
echo "+memory" > "${CGPATH}/L/cgroup.subtree_control"
mkdir "${CGPATH}/A/b" "${CGPATH}/A/C" "${CGPATH}/A/D" "${CGPATH}/A/e"
mkdir "${CGPATH}/F/g" "${CGPATH}/F/H"
mkdir "${CGPATH}/i/j" "${CGPATH}/i/K"
mkdir "${CGPATH}/L/m" "${CGPATH}/L/N"
echo 2G > "${CGPATH}/A/memory.${LIM}"
echo 3G > "${CGPATH}/A/b/memory.${LIM}"
echo 1G > "${CGPATH}/A/C/memory.${LIM}"
echo 0 > "${CGPATH}/A/D/memory.${LIM}"
echo 10G > "${CGPATH}/A/e/memory.${LIM}"
echo 2G > "${CGPATH}/F/memory.${LIM}"
echo 3G > "${CGPATH}/F/g/memory.${LIM}"
echo 1G > "${CGPATH}/F/H/memory.${LIM}"
echo 5G > "${CGPATH}/i/memory.${LIM}"
echo 3G > "${CGPATH}/i/j/memory.${LIM}"
echo 1G > "${CGPATH}/i/K/memory.${LIM}"
echo 2G > "${CGPATH}/L/memory.${LIM}"
echo 4G > "${CGPATH}/L/memory.max"
echo 3G > "${CGPATH}/L/m/memory.${LIM}"
echo 1G > "${CGPATH}/L/N/memory.${LIM}"
vmtouch A/b /file_b 2G
vmtouch A/C /file_C 2G
vmtouch A/D /file_D 2G
vmtouch F/g /file_g 2G
vmtouch F/H /file_H 2G
vmtouch i/j /file_j 2G
vmtouch i/K /file_K 2G
vmtouch L/m /file_m 2G
vmtouch L/N /file_N 2G
vmtouch2 "$ANTAGONIST" /file_ant 150G
echo
echo "after $ANTAGONIST antagonist"
dump
rmdir "${CGPATH}/A/b" "${CGPATH}/A/C" "${CGPATH}/A/D" "${CGPATH}/A/e"
rmdir "${CGPATH}/F/g" "${CGPATH}/F/H"
rmdir "${CGPATH}/i/j" "${CGPATH}/i/K"
rmdir "${CGPATH}/L/m" "${CGPATH}/L/N"
rmdir "${CGPATH}/A" "${CGPATH}/F" "${CGPATH}/i" "${CGPATH}/L"
rm /file_ant /file_b /file_C /file_D /file_g /file_H /file_j /file_K
Greg Thelen (2):
memcg: fix memory.low
memcg: add memory.min
include/linux/memcontrol.h | 8 +++++
mm/memcontrol.c | 70 ++++++++++++++++++++++++++++++++++----
mm/vmscan.c | 3 ++
3 files changed, 75 insertions(+), 6 deletions(-)
--
2.17.0.484.g0c8726318c-goog
