On Thu, Apr 15, 2021 at 8:35 AM Dave Hansen <dave.hansen@xxxxxxxxx> wrote: > > This can help enable more flexible demotion policies to be > > configured, such as to allow a cgroup to allocate from all fast tier > > nodes, but only demote to a local slow tier node. Such a policy can > > reduce memory stranding at the fast tier (compared to if memory > > hardwall is used) and still allow demotion from all fast tier nodes > > without incurring the expensive random accesses to the demoted pages > > if they were demoted to remote slow tier nodes. > > Could you explain this stranding effect in a bit more detail? I'm not > quite following. By memory stranding, I mean that memory on a machine (or a NUMA node) cannot be utilized even under extremely high work loads. Memory stranding happens usually due to mismatches between job/machine shapes as well as resource fragmentation resulted from bin-packing scheduling. It is an important problem for cloud resource efficiency. If NUMA hardwalling is used, we effectively split a single machine into multiple smaller machines based on NUMA nodes. This changes the machine shapes and also makes memory more fragmented, which can lead to more memory being stranded. Here is a simple example: Suppose that each machine has 2 NUMA nodes, each with 4 cores and 5GB RAM, and all the jobs have the shape of 2 CPUs and 3GB memory. Without NUMA memory hardwalling, we can pack 3 jobs onto each machine, which leaves 1GB memory and 2 cores in stranding. However, with NUMA memory hardwalling enabled, we can then only pack 2 jobs onto each machine (one job on each NUMA node), which increases the resource stranding to 4GB memory and 4 cores.
