On Wed, 17 Aug 2011 09:15:04 -0700
Greg Thelen <gthelen@xxxxxxxxxx> wrote:
> Introduce memcg routines to assist in per-memcg dirty page management:
>
> - mem_cgroup_balance_dirty_pages() walks a memcg hierarchy comparing
> dirty memory usage against memcg foreground and background thresholds.
> If an over-background-threshold memcg is found, then per-memcg
> background writeback is queued. Per-memcg writeback differs from
> classic, non-memcg, per bdi writeback by setting the new
> writeback_control.for_cgroup bit.
>
> If an over-foreground-threshold memcg is found, then foreground
> writeout occurs. When performing foreground writeout, first consider
> inodes exclusive to the memcg. If unable to make enough progress,
> then consider inodes shared between memcg. Such cross-memcg inode
> sharing likely to be rare in situations that use per-cgroup memory
> isolation. So the approach tries to handle the common case well
> without falling over in cases where such sharing exists. This routine
> is used by balance_dirty_pages() in a later change.
>
> - mem_cgroup_hierarchical_dirty_info() returns the dirty memory usage
> and limits of the memcg closest to (or over) its dirty limit. This
> will be used by throttle_vm_writeout() in a latter change.
>
> Signed-off-by: Greg Thelen <gthelen@xxxxxxxxxx>
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
Comparing page-writebakc.c, I have some questions.
> +/*
> + * This routine must be called periodically by processes which generate dirty
> + * pages. It considers the dirty pages usage and thresholds of the current
> + * cgroup and (depending if hierarchical accounting is enabled) ancestral memcg.
> + * If any of the considered memcg are over their background dirty limit, then
> + * background writeback is queued. If any are over the foreground dirty limit
> + * then the dirtying task is throttled while writing dirty data. The per-memcg
> + * dirty limits checked by this routine are distinct from either the per-system,
> + * per-bdi, or per-task limits considered by balance_dirty_pages().
> + *
> + * Example hierarchy:
> + * root
> + * A B
> + * A1 A2 B1
> + * A11 A12 A21 A22
> + *
> + * Assume that mem_cgroup_balance_dirty_pages() is called on A11. This routine
> + * starts at A11 walking upwards towards the root. If A11 is over dirty limit,
> + * then writeback A11 inodes until under limit. Next check A1, if over limit
> + * then write A1,A11,A12. Then check A. If A is over A limit, then invoke
> + * writeback on A* until A is under A limit.
> + */
> +void mem_cgroup_balance_dirty_pages(struct address_space *mapping,
> + unsigned long write_chunk)
> +{
> + struct backing_dev_info *bdi = mapping->backing_dev_info;
> + struct mem_cgroup *memcg;
> + struct mem_cgroup *ref_memcg;
> + struct dirty_info info;
> + unsigned long nr_reclaimable;
> + unsigned long nr_written;
> + unsigned long sys_available_mem;
> + unsigned long pause = 1;
> + unsigned short id;
> + bool over;
> + bool shared_inodes;
> +
> + if (mem_cgroup_disabled())
> + return;
> +
> + sys_available_mem = determine_dirtyable_memory();
> +
> + /* reference the memcg so it is not deleted during this routine */
> + rcu_read_lock();
> + memcg = mem_cgroup_from_task(current);
> + if (memcg && mem_cgroup_is_root(memcg))
> + memcg = NULL;
> + if (memcg)
> + css_get(&memcg->css);
> + rcu_read_unlock();
> + ref_memcg = memcg;
> +
> + /* balance entire ancestry of current's memcg. */
> + for (; mem_cgroup_has_dirty_limit(memcg);
> + memcg = parent_mem_cgroup(memcg)) {
> + id = css_id(&memcg->css);
> +
> + /*
> + * Keep throttling and writing inode data so long as memcg is
> + * over its dirty limit. Inode being written by multiple memcg
> + * (aka shared_inodes) cannot easily be attributed a particular
> + * memcg. Shared inodes are thought to be much rarer than
> + * shared inodes. First try to satisfy this memcg's dirty
> + * limits using non-shared inodes.
> + */
> + for (shared_inodes = false; ; ) {
> + /*
> + * if memcg is under dirty limit, then break from
> + * throttling loop.
> + */
> + mem_cgroup_dirty_info(sys_available_mem, memcg, &info);
> + nr_reclaimable = dirty_info_reclaimable(&info);
> + over = nr_reclaimable > info.dirty_thresh;
> + trace_mem_cgroup_consider_fg_writeback(
> + id, bdi, nr_reclaimable, info.dirty_thresh,
> + over);
> + if (!over)
> + break;
> +
> + nr_written = writeback_inodes_wb(&bdi->wb, write_chunk,
> + memcg, shared_inodes);
> + trace_mem_cgroup_fg_writeback(write_chunk, nr_written,
> + id, shared_inodes);
> + /* if no progress, then consider shared inodes */
> + if ((nr_written == 0) && !shared_inodes) {
> + trace_mem_cgroup_enable_shared_writeback(id);
> + shared_inodes = true;
> + }
in page-writeback.c
if (pages_written >= write_chunk)
break; /* We've done our duty */
write_chunk(ratelimit) is used. Can't we make use of this threshold ?
> +
> + __set_current_state(TASK_UNINTERRUPTIBLE);
> + io_schedule_timeout(pause);
> +
How do you think about MAX_PAUSE/PASS_GOOD ?
==
/*
* max-pause area. If dirty exceeded but still within this
* area, no need to sleep for more than 200ms: (a) 8 pages per
* 200ms is typically more than enough to curb heavy dirtiers;
* (b) the pause time limit makes the dirtiers more responsive.
*/
if (nr_dirty < dirty_thresh +
dirty_thresh / DIRTY_MAXPAUSE_AREA &&
time_after(jiffies, start_time + MAX_PAUSE))
break;
/*
* pass-good area. When some bdi gets blocked (eg. NFS server
* not responding), or write bandwidth dropped dramatically due
* to concurrent reads, or dirty threshold suddenly dropped and
* the dirty pages cannot be brought down anytime soon (eg. on
* slow USB stick), at least let go of the good bdi's.
*/
if (nr_dirty < dirty_thresh +
dirty_thresh / DIRTY_PASSGOOD_AREA &&
bdi_dirty < bdi_thresh)
break;
==
Thanks,
-Kame
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
