Currently, all calls to flush memcg stats use the atomic variant for rstat flushing, cgroup_rstat_flush_irqsafe(), which keeps interrupts disabled throughout flushing and does not sleep. Flushing stats is an expensive operation, and we should avoid doing it atomically where possible. Otherwise, we may end up doing a lot of work without rescheduling and with interrupts disabled unnecessarily. Patches 1 and 2 are cleanups requested during reviews of prior versions of this series. Patch 3 makes sure we never try to flush from within an irq context, and patch 4 adds a WARN_ON_ONCE() to make sure we catch any violations. Patches 5 to 8 introduce separate variants of mem_cgroup_flush_stats() for atomic and non-atomic flushing, and make sure we only flush the stats atomically when necessary. Patch 9 is a slightly tangential optimization that limits the work done by rstat flushing in some scenarios. RFC -> v1: - Dropped patch 1 that attempted to make the global rstat lock a non-irq lock, will follow up on that separetly (Shakeel). - Dropped stats_flush_lock entirely, replaced by an atomic (Johannes). - Renamed cgroup_rstat_flush_irqsafe() to cgroup_rstat_flush_atomic() instead of removing it (Johannes). - Added a patch to rename mem_cgroup_flush_stats_delayed() to mem_cgroup_flush_stats_ratelimited() (Johannes). - Separate APIs for flushing memcg stats in atomic and non-atomic contexts instead of a boolean argument (Johannes). - Added patches 3 & 4 to make sure we never flush from irq context (Shakeel & Johannes). Yosry Ahmed (9): cgroup: rename cgroup_rstat_flush_"irqsafe" to "atomic" memcg: rename mem_cgroup_flush_stats_"delayed" to "ratelimited" memcg: do not flush stats in irq context cgroup: rstat: add WARN_ON_ONCE() if flushing outside task context memcg: replace stats_flush_lock with an atomic memcg: sleep during flushing stats in safe contexts workingset: memcg: sleep when flushing stats in workingset_refault() vmscan: memcg: sleep when flushing stats during reclaim memcg: do not modify rstat tree for zero updates include/linux/cgroup.h | 2 +- include/linux/memcontrol.h | 9 +++- kernel/cgroup/rstat.c | 6 ++- mm/memcontrol.c | 86 ++++++++++++++++++++++++++++++++------ mm/workingset.c | 4 +- 5 files changed, 87 insertions(+), 20 deletions(-) -- 2.40.0.348.gf938b09366-goog
