On Mon, 25 Apr 2011 21:59:06 -0700
Ying Han <yinghan@xxxxxxxxxx> wrote:
> On Mon, Apr 25, 2011 at 2:36 AM, KAMEZAWA Hiroyuki
> <kamezawa.hiroyu@xxxxxxxxxxxxxx> wrote:
> > Following patch will chagnge the logic. This is a core.
> > ==
> > This is the main loop of per-memcg background reclaim which is implemented in
> > function balance_mem_cgroup_pgdat().
> >
> > The function performs a priority loop similar to global reclaim. During each
> > iteration it frees memory from a selected victim node.
> > After reclaiming enough pages or scanning enough pages, it returns and find
> > next work with round-robin.
> >
> > changelog v8b..v7
> > 1. reworked for using work_queue rather than threads.
> > 2. changed shrink_mem_cgroup algorithm to fit workqueue. In short, avoid
> > Â long running and allow quick round-robin and unnecessary write page.
> > Â When a thread make pages dirty continuously, write back them by flusher
> > Â is far faster than writeback by background reclaim. This detail will
> > Â be fixed when dirty_ratio implemented. The logic around this will be
> > Â revisited in following patche.
> >
> > Signed-off-by: Ying Han <yinghan@xxxxxxxxxx>
> > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
> > ---
> > Âinclude/linux/memcontrol.h | Â 11 ++++
> > Âmm/memcontrol.c      Â|  44 ++++++++++++++---
> > Âmm/vmscan.c        Â| Â115 +++++++++++++++++++++++++++++++++++++++++++++
> > Â3 files changed, 162 insertions(+), 8 deletions(-)
> >
> > Index: memcg/include/linux/memcontrol.h
> > ===================================================================
> > --- memcg.orig/include/linux/memcontrol.h
> > +++ memcg/include/linux/memcontrol.h
> > @@ -89,6 +89,8 @@ extern int mem_cgroup_last_scanned_node(
> > Âextern int mem_cgroup_select_victim_node(struct mem_cgroup *mem,
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Âconst nodemask_t *nodes);
> >
> > +unsigned long shrink_mem_cgroup(struct mem_cgroup *mem);
> > +
> > Âstatic inline
> > Âint mm_match_cgroup(const struct mm_struct *mm, const struct mem_cgroup *cgroup)
> > Â{
> > @@ -112,6 +114,9 @@ extern void mem_cgroup_end_migration(str
> > Â*/
> > Âint mem_cgroup_inactive_anon_is_low(struct mem_cgroup *memcg);
> > Âint mem_cgroup_inactive_file_is_low(struct mem_cgroup *memcg);
> > +unsigned int mem_cgroup_swappiness(struct mem_cgroup *memcg);
> > +unsigned long mem_cgroup_zone_reclaimable_pages(struct mem_cgroup *memcg,
> > + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â int nid, int zone_idx);
> > Âunsigned long mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg,
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â struct zone *zone,
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â enum lru_list lru);
> > @@ -310,6 +315,12 @@ mem_cgroup_inactive_file_is_low(struct m
> > Â}
> >
> > Âstatic inline unsigned long
> > +mem_cgroup_zone_reclaimable_pages(struct mem_cgroup *memcg, int nid, int zone_idx)
> > +{
> > + Â Â Â return 0;
> > +}
> > +
> > +static inline unsigned long
> > Âmem_cgroup_zone_nr_pages(struct mem_cgroup *memcg, struct zone *zone,
> > Â Â Â Â Â Â Â Â Â Â Â Â enum lru_list lru)
> > Â{
> > Index: memcg/mm/memcontrol.c
> > ===================================================================
> > --- memcg.orig/mm/memcontrol.c
> > +++ memcg/mm/memcontrol.c
> > @@ -1166,6 +1166,23 @@ int mem_cgroup_inactive_file_is_low(stru
> > Â Â Â Âreturn (active > inactive);
> > Â}
> >
> > +unsigned long mem_cgroup_zone_reclaimable_pages(struct mem_cgroup *memcg,
> > + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â int nid, int zone_idx)
> > +{
> > + Â Â Â int nr;
> > + Â Â Â struct mem_cgroup_per_zone *mz =
> > + Â Â Â Â Â Â Â mem_cgroup_zoneinfo(memcg, nid, zone_idx);
> > +
> > + Â Â Â nr = MEM_CGROUP_ZSTAT(mz, NR_ACTIVE_FILE) +
> > + Â Â Â Â Â ÂMEM_CGROUP_ZSTAT(mz, NR_INACTIVE_FILE);
> > +
> > + Â Â Â if (nr_swap_pages > 0)
> > + Â Â Â Â Â Â Â nr += MEM_CGROUP_ZSTAT(mz, NR_ACTIVE_ANON) +
> > + Â Â Â Â Â Â Â Â Â Â MEM_CGROUP_ZSTAT(mz, NR_INACTIVE_ANON);
> > +
> > + Â Â Â return nr;
> > +}
> > +
> > Âunsigned long mem_cgroup_zone_nr_pages(struct mem_cgroup *memcg,
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â struct zone *zone,
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â enum lru_list lru)
> > @@ -1286,7 +1303,7 @@ static unsigned long mem_cgroup_margin(s
> > Â Â Â Âreturn margin >> PAGE_SHIFT;
> > Â}
> >
> > -static unsigned int get_swappiness(struct mem_cgroup *memcg)
> > +unsigned int mem_cgroup_swappiness(struct mem_cgroup *memcg)
> > Â{
> > Â Â Â Âstruct cgroup *cgrp = memcg->css.cgroup;
> >
> > @@ -1595,14 +1612,15 @@ static int mem_cgroup_hierarchical_recla
> > Â Â Â Â Â Â Â Â/* we use swappiness of local cgroup */
> > Â Â Â Â Â Â Â Âif (check_soft) {
> > Â Â Â Â Â Â Â Â Â Â Â Âret = mem_cgroup_shrink_node_zone(victim, gfp_mask,
> > - Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â noswap, get_swappiness(victim), zone,
> > + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â noswap, mem_cgroup_swappiness(victim), zone,
> > Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â&nr_scanned);
> > Â Â Â Â Â Â Â Â Â Â Â Â*total_scanned += nr_scanned;
> > Â Â Â Â Â Â Â Â Â Â Â Âmem_cgroup_soft_steal(victim, ret);
> > Â Â Â Â Â Â Â Â Â Â Â Âmem_cgroup_soft_scan(victim, nr_scanned);
> > Â Â Â Â Â Â Â Â} else
> > Â Â Â Â Â Â Â Â Â Â Â Âret = try_to_free_mem_cgroup_pages(victim, gfp_mask,
> > - Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â noswap, get_swappiness(victim));
> > + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â noswap,
> > + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â mem_cgroup_swappiness(victim));
> > Â Â Â Â Â Â Â Âcss_put(&victim->css);
> > Â Â Â Â Â Â Â Â/*
> > Â Â Â Â Â Â Â Â * At shrinking usage, we can't check we should stop here or
> > @@ -1628,15 +1646,25 @@ static int mem_cgroup_hierarchical_recla
> > Âint
> > Âmem_cgroup_select_victim_node(struct mem_cgroup *mem, const nodemask_t *nodes)
> > Â{
> > - Â Â Â int next_nid;
> > + Â Â Â int next_nid, i;
> > Â Â Â Âint last_scanned;
> >
> > Â Â Â Âlast_scanned = mem->last_scanned_node;
> > - Â Â Â next_nid = next_node(last_scanned, *nodes);
> > + Â Â Â next_nid = last_scanned;
> > +rescan:
> > + Â Â Â next_nid = next_node(next_nid, *nodes);
> >
> > Â Â Â Âif (next_nid == MAX_NUMNODES)
> > Â Â Â Â Â Â Â Ânext_nid = first_node(*nodes);
> >
> > + Â Â Â /* If no page on this node, skip */
> > + Â Â Â for (i = 0; i < MAX_NR_ZONES; i++)
> > + Â Â Â Â Â Â Â if (mem_cgroup_zone_reclaimable_pages(mem, next_nid, i))
> > + Â Â Â Â Â Â Â Â Â Â Â break;
> > +
> > + Â Â Â if (next_nid != last_scanned && (i == MAX_NR_ZONES))
> > + Â Â Â Â Â Â Â goto rescan;
> > +
> > Â Â Â Âmem->last_scanned_node = next_nid;
> >
> > Â Â Â Âreturn next_nid;
> > @@ -3649,7 +3677,7 @@ try_to_free:
> > Â Â Â Â Â Â Â Â Â Â Â Âgoto out;
> > Â Â Â Â Â Â Â Â}
> > Â Â Â Â Â Â Â Âprogress = try_to_free_mem_cgroup_pages(mem, GFP_KERNEL,
> > - Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â false, get_swappiness(mem));
> > + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â false, mem_cgroup_swappiness(mem));
> > Â Â Â Â Â Â Â Âif (!progress) {
> > Â Â Â Â Â Â Â Â Â Â Â Ânr_retries--;
> > Â Â Â Â Â Â Â Â Â Â Â Â/* maybe some writeback is necessary */
> > @@ -4073,7 +4101,7 @@ static u64 mem_cgroup_swappiness_read(st
> > Â{
> > Â Â Â Âstruct mem_cgroup *memcg = mem_cgroup_from_cont(cgrp);
> >
> > - Â Â Â return get_swappiness(memcg);
> > + Â Â Â return mem_cgroup_swappiness(memcg);
> > Â}
> >
> > Âstatic int mem_cgroup_swappiness_write(struct cgroup *cgrp, struct cftype *cft,
> > @@ -4849,7 +4877,7 @@ mem_cgroup_create(struct cgroup_subsys *
> > Â Â Â ÂINIT_LIST_HEAD(&mem->oom_notify);
> >
> > Â Â Â Âif (parent)
> > - Â Â Â Â Â Â Â mem->swappiness = get_swappiness(parent);
> > + Â Â Â Â Â Â Â mem->swappiness = mem_cgroup_swappiness(parent);
> > Â Â Â Âatomic_set(&mem->refcnt, 1);
> > Â Â Â Âmem->move_charge_at_immigrate = 0;
> > Â Â Â Âmutex_init(&mem->thresholds_lock);
> > Index: memcg/mm/vmscan.c
> > ===================================================================
> > --- memcg.orig/mm/vmscan.c
> > +++ memcg/mm/vmscan.c
> > @@ -42,6 +42,7 @@
> > Â#include <linux/delayacct.h>
> > Â#include <linux/sysctl.h>
> > Â#include <linux/oom.h>
> > +#include <linux/res_counter.h>
> >
> > Â#include <asm/tlbflush.h>
> > Â#include <asm/div64.h>
> > @@ -2308,6 +2309,120 @@ static bool sleeping_prematurely(pg_data
> > Â Â Â Â Â Â Â Âreturn !all_zones_ok;
> > Â}
> >
> > +#ifdef CONFIG_CGROUP_MEM_RES_CTLR
> > +/*
> > + * The function is used for per-memcg LRU. It scanns all the zones of the
> > + * node and returns the nr_scanned and nr_reclaimed.
> > + */
> > +/*
> > + * Limit of scanning per iteration. For round-robin.
> > + */
> > +#define MEMCG_BGSCAN_LIMIT Â Â (2048)
> > +
> > +static void
> > +shrink_memcg_node(int nid, int priority, struct scan_control *sc)
> > +{
> > + Â Â Â unsigned long total_scanned = 0;
> > + Â Â Â struct mem_cgroup *mem_cont = sc->mem_cgroup;
> > + Â Â Â int i;
> > +
> > + Â Â Â /*
> > + Â Â Â Â* This dma->highmem order is consistant with global reclaim.
> > + Â Â Â Â* We do this because the page allocator works in the opposite
> > + Â Â Â Â* direction although memcg user pages are mostly allocated at
> > + Â Â Â Â* highmem.
> > + Â Â Â Â*/
> > + Â Â Â for (i = 0;
> > + Â Â Â Â Â Â(i < NODE_DATA(nid)->nr_zones) &&
> > + Â Â Â Â Â Â(total_scanned < MEMCG_BGSCAN_LIMIT);
> > + Â Â Â Â Â Âi++) {
> > + Â Â Â Â Â Â Â struct zone *zone = NODE_DATA(nid)->node_zones + i;
> > + Â Â Â Â Â Â Â struct zone_reclaim_stat *zrs;
> > + Â Â Â Â Â Â Â unsigned long scan, rotate;
> > +
> > + Â Â Â Â Â Â Â if (!populated_zone(zone))
> > + Â Â Â Â Â Â Â Â Â Â Â continue;
> > + Â Â Â Â Â Â Â scan = mem_cgroup_zone_reclaimable_pages(mem_cont, nid, i);
> > + Â Â Â Â Â Â Â if (!scan)
> > + Â Â Â Â Â Â Â Â Â Â Â continue;
> > + Â Â Â Â Â Â Â /* If recent memory reclaim on this zone doesn't get good */
> > + Â Â Â Â Â Â Â zrs = get_reclaim_stat(zone, sc);
> > + Â Â Â Â Â Â Â scan = zrs->recent_scanned[0] + zrs->recent_scanned[1];
> > + Â Â Â Â Â Â Â rotate = zrs->recent_rotated[0] + zrs->recent_rotated[1];
> > +
> > + Â Â Â Â Â Â Â if (rotate > scan/2)
> > + Â Â Â Â Â Â Â Â Â Â Â sc->may_writepage = 1;
> > +
> > + Â Â Â Â Â Â Â sc->nr_scanned = 0;
> > + Â Â Â Â Â Â Â shrink_zone(priority, zone, sc);
> > + Â Â Â Â Â Â Â total_scanned += sc->nr_scanned;
> > + Â Â Â Â Â Â Â sc->may_writepage = 0;
> > + Â Â Â }
> > + Â Â Â sc->nr_scanned = total_scanned;
> > +}
>
> I see the MEMCG_BGSCAN_LIMIT is a newly defined macro from previous
> post. So, now the number of pages to scan is capped on 2k for each
> memcg, and does it make difference on big vs small cgroup?
>
Now, no difference. One reason is because low_watermark - high_watermark is
limited to 4MB, at most. It should be static 4MB in many cases and 2048 pages
is for scanning 8MB, twice of low_wmark - high_wmark. Another reason is
that I didn't have enough time for considering to tune this.
By MEMCG_BGSCAN_LIMIT, round-robin can be simply fair and I think it's a
good start point.
If memory eater enough slow (because the threads needs to do some
work on allocated memory), this shrink_mem_cgroup() works fine and
helps to avoid hitting limit. Here, the amount of dirty pages is troublesome.
The penaly for cpu eating (hard-to-reclaim) cgroup is given by 'delay'.
(see patch 7.) This patch's congestion_wait is too bad and will be replaced
in patch 7 as 'delay'. In short, if memcg scanning seems to be not successful,
it gets HZ/10 delay until the next work.
If we have dirty_ratio + I/O less dirty throttling, I think we'll see much
better fairness on this watermark reclaim round robin.
Thanks,
-Kame
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxxx For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>