On Thu, Mar 05, 2020 at 09:55:54AM -0500, Tejun Heo wrote: > Changing memory limits dynamically can lead to pretty abrupt system > behaviors depending on how big the swing is but memory.low and io/cpu > weights should behave fine. A couple more things which might be helpful. * memory.min/low are pretty forgiving. Semantically what it tells the kernel is "reclaim this guy as if the protected amount isn't being consumed" - if a cgroup consumes 8G and has 4G protection, it'd get the same reclaim pressure as a sibling whose consuming 4G without protection. While the range of "good" configuration is pretty wide, you can definitely push it too far to the point the rest of the system has to compete too hard for memory. In practice, setting memory protection to something like 50-75% of expected allocation seems to work well - it provides ample protection while allowing the system to be flexible when it needs to be. One important thing that we learned is that as resource configuration gets more rigid, it can also become more brittle. * As for io.weight (and cpu.weight too), while prioritization is meaningful, what matters the most is avoiding situations where one consumer overwhelms the device. Simply configuring io.cost correctly and enabling it with default weights may achieve majority of the benefits even without specific weight configurations. Please note that IO control has quite a bit of requirements to function correctly - it currently works well only on btrfs on physical (not md or dm) devices with all other IO controllers and wbt disabled. Hopefully, we'll be able to relax the requirements in the future but we aren't there yet. With both memory and IO set up and oomd watching out for swap depletion, our configurations show almost complete resource isolation where no matter what we do in unprotected portion of the system it doesn't affect the performance of the protected portion much even when the protected portion is running resource hungry latency sensitive workloads. Thanks. -- tejun
