On Sat, Apr 16, 2011 at 8:23 AM, Ying Han <yinghan@xxxxxxxxxx> wrote:
> This is the main loop of per-memcg background reclaim which is implemented in
> function balance_mem_cgroup_pgdat().
>
> The function performs a priority loop similar to global reclaim. During each
> iteration it invokes balance_pgdat_node() for all nodes on the system, which
> is another new function performs background reclaim per node. After reclaiming
> each node, it checks mem_cgroup_watermark_ok() and breaks the priority loop if
> it returns true.
>
> changelog v5..v4:
> 1. remove duplicate check on nodes_empty()
> 2. add logic to check if the per-memcg lru is empty on the zone.
> 3. make per-memcg kswapd to reclaim SWAP_CLUSTER_MAX per zone. It make senses
> since it helps to balance the pressure across zones within the memcg.
>
> changelog v4..v3:
> 1. split the select_victim_node and zone_unreclaimable to a seperate patches
> 2. remove the logic tries to do zone balancing.
>
> changelog v3..v2:
> 1. change mz->all_unreclaimable to be boolean.
> 2. define ZONE_RECLAIMABLE_RATE macro shared by zone and per-memcg reclaim.
> 3. some more clean-up.
>
> changelog v2..v1:
> 1. move the per-memcg per-zone clear_unreclaimable into uncharge stage.
> 2. shared the kswapd_run/kswapd_stop for per-memcg and global background
> reclaim.
> 3. name the per-memcg memcg as "memcg-id" (css->id). And the global kswapd
> keeps the same name.
> 4. fix a race on kswapd_stop while the per-memcg-per-zone info could be accessed
> after freeing.
> 5. add the fairness in zonelist where memcg remember the last zone reclaimed
> from.
>
> Signed-off-by: Ying Han <yinghan@xxxxxxxxxx>
> ---
> Âmm/vmscan.c | Â157 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Â1 files changed, 157 insertions(+), 0 deletions(-)
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 06036d2..39e6300 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -47,6 +47,8 @@
>
> Â#include <linux/swapops.h>
>
> +#include <linux/res_counter.h>
> +
> Â#include "internal.h"
>
> Â#define CREATE_TRACE_POINTS
> @@ -111,6 +113,8 @@ struct scan_control {
> Â Â Â Â * are scanned.
> Â Â Â Â */
>    Ânodemask_t   Â*nodemask;
> +
> + Â Â Â int priority;
> Â};
>
> Â#define lru_to_page(_head) (list_entry((_head)->prev, struct page, lru))
> @@ -2631,11 +2635,164 @@ static void kswapd_try_to_sleep(struct kswapd *kswapd_p, int order,
> Â Â Â Âfinish_wait(wait_h, &wait);
> Â}
>
> +#ifdef CONFIG_CGROUP_MEM_RES_CTLR
> +/*
> + * The function is used for per-memcg LRU. It scanns all the zones of the
> + * node and returns the nr_scanned and nr_reclaimed.
> + */
> +static void balance_pgdat_node(pg_data_t *pgdat, int order,
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â struct scan_control *sc)
> +{
> + Â Â Â int i;
> + Â Â Â unsigned long total_scanned = 0;
> + Â Â Â struct mem_cgroup *mem_cont = sc->mem_cgroup;
> + Â Â Â int priority = sc->priority;
> + Â Â Â enum lru_list l;
> +
> + Â Â Â /*
> + Â Â Â Â* This dma->highmem order is consistant with global reclaim.
> + Â Â Â Â* We do this because the page allocator works in the opposite
> + Â Â Â Â* direction although memcg user pages are mostly allocated at
> + Â Â Â Â* highmem.
> + Â Â Â Â*/
> + Â Â Â for (i = 0; i < pgdat->nr_zones; i++) {
> + Â Â Â Â Â Â Â struct zone *zone = pgdat->node_zones + i;
> + Â Â Â Â Â Â Â unsigned long scan = 0;
> +
> + Â Â Â Â Â Â Â for_each_evictable_lru(l)
> + Â Â Â Â Â Â Â Â Â Â Â scan += mem_cgroup_zone_nr_pages(mem_cont, zone, l);
> +
> + Â Â Â Â Â Â Â if (!populated_zone(zone) || !scan)
> + Â Â Â Â Â Â Â Â Â Â Â continue;
Do we really need this double check?
Isn't only _scan_ check enough?
And shouldn't we consider non-swap case?
> +
> + Â Â Â Â Â Â Â sc->nr_scanned = 0;
> + Â Â Â Â Â Â Â shrink_zone(priority, zone, sc);
> + Â Â Â Â Â Â Â total_scanned += sc->nr_scanned;
> +
> + Â Â Â Â Â Â Â /*
> + Â Â Â Â Â Â Â Â* If we've done a decent amount of scanning and
> + Â Â Â Â Â Â Â Â* the reclaim ratio is low, start doing writepage
> + Â Â Â Â Â Â Â Â* even in laptop mode
> + Â Â Â Â Â Â Â Â*/
> + Â Â Â Â Â Â Â if (total_scanned > SWAP_CLUSTER_MAX * 2 &&
> + Â Â Â Â Â Â Â Â Â total_scanned > sc->nr_reclaimed + sc->nr_reclaimed / 2) {
> + Â Â Â Â Â Â Â Â Â Â Â sc->may_writepage = 1;
I don't want to add more random write any more although we don't have
a trouble of real memory shortage.
Do you have any reason to reclaim memory urgently as writing dirty pages?
Maybe if we wait a little bit of time, flusher would write out the page.
> + Â Â Â Â Â Â Â }
> + Â Â Â }
> +
> + Â Â Â sc->nr_scanned = total_scanned;
> + Â Â Â return;
unnecessary return.
> +}
> +
> +/*
> + * Per cgroup background reclaim.
> + * TODO: Take off the order since memcg always do order 0
> + */
> +static unsigned long balance_mem_cgroup_pgdat(struct mem_cgroup *mem_cont,
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â int order)
> +{
> + Â Â Â int i, nid;
> + Â Â Â int start_node;
> + Â Â Â int priority;
> + Â Â Â bool wmark_ok;
> + Â Â Â int loop;
> + Â Â Â pg_data_t *pgdat;
> + Â Â Â nodemask_t do_nodes;
> + Â Â Â unsigned long total_scanned;
> + Â Â Â struct scan_control sc = {
> + Â Â Â Â Â Â Â .gfp_mask = GFP_KERNEL,
> + Â Â Â Â Â Â Â .may_unmap = 1,
> + Â Â Â Â Â Â Â .may_swap = 1,
> + Â Â Â Â Â Â Â .nr_to_reclaim = SWAP_CLUSTER_MAX,
> + Â Â Â Â Â Â Â .swappiness = vm_swappiness,
> + Â Â Â Â Â Â Â .order = order,
> + Â Â Â Â Â Â Â .mem_cgroup = mem_cont,
> + Â Â Â };
> +
> +loop_again:
> + Â Â Â do_nodes = NODE_MASK_NONE;
> + Â Â Â sc.may_writepage = !laptop_mode;
I think it depends on urgency(ie, priority, reclaim
ratio(#reclaim/#scanning) or something), not laptop_mode in case of
memcg.
As I said earlier,it wold be better to avoid random write.
> + Â Â Â sc.nr_reclaimed = 0;
> + Â Â Â total_scanned = 0;
> +
> + Â Â Â for (priority = DEF_PRIORITY; priority >= 0; priority--) {
> + Â Â Â Â Â Â Â sc.priority = priority;
> + Â Â Â Â Â Â Â wmark_ok = false;
> + Â Â Â Â Â Â Â loop = 0;
> +
> + Â Â Â Â Â Â Â /* The swap token gets in the way of swapout... */
> + Â Â Â Â Â Â Â if (!priority)
> + Â Â Â Â Â Â Â Â Â Â Â disable_swap_token();
> +
> + Â Â Â Â Â Â Â if (priority == DEF_PRIORITY)
> + Â Â Â Â Â Â Â Â Â Â Â do_nodes = node_states[N_ONLINE];
> +
> + Â Â Â Â Â Â Â while (1) {
> + Â Â Â Â Â Â Â Â Â Â Â nid = mem_cgroup_select_victim_node(mem_cont,
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â &do_nodes);
> +
> + Â Â Â Â Â Â Â Â Â Â Â /* Indicate we have cycled the nodelist once
Fix comment style.
> + Â Â Â Â Â Â Â Â Â Â Â Â* TODO: we might add MAX_RECLAIM_LOOP for preventing
> + Â Â Â Â Â Â Â Â Â Â Â Â* kswapd burning cpu cycles.
> + Â Â Â Â Â Â Â Â Â Â Â Â*/
> + Â Â Â Â Â Â Â Â Â Â Â if (loop == 0) {
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â start_node = nid;
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â loop++;
> + Â Â Â Â Â Â Â Â Â Â Â } else if (nid == start_node)
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â break;
> +
> + Â Â Â Â Â Â Â Â Â Â Â pgdat = NODE_DATA(nid);
> + Â Â Â Â Â Â Â Â Â Â Â balance_pgdat_node(pgdat, order, &sc);
> + Â Â Â Â Â Â Â Â Â Â Â total_scanned += sc.nr_scanned;
> +
> + Â Â Â Â Â Â Â Â Â Â Â /* Set the node which has at least
Fix comment style.
> + Â Â Â Â Â Â Â Â Â Â Â Â* one reclaimable zone
> + Â Â Â Â Â Â Â Â Â Â Â Â*/
> + Â Â Â Â Â Â Â Â Â Â Â for (i = pgdat->nr_zones - 1; i >= 0; i--) {
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â struct zone *zone = pgdat->node_zones + i;
> +
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â if (!populated_zone(zone))
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â continue;
> + Â Â Â Â Â Â Â Â Â Â Â }
I can't understand your comment and logic.
The comment mentioned reclaimable zone but the logic checks just
populated_zone. What's meaning?
> + Â Â Â Â Â Â Â Â Â Â Â if (i < 0)
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â node_clear(nid, do_nodes);
> +
> + Â Â Â Â Â Â Â Â Â Â Â if (mem_cgroup_watermark_ok(mem_cont,
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â CHARGE_WMARK_HIGH)) {
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â wmark_ok = true;
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â goto out;
> + Â Â Â Â Â Â Â Â Â Â Â }
> +
> + Â Â Â Â Â Â Â Â Â Â Â if (nodes_empty(do_nodes)) {
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â wmark_ok = true;
> + Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â goto out;
> + Â Â Â Â Â Â Â Â Â Â Â }
> + Â Â Â Â Â Â Â }
> +
> + Â Â Â Â Â Â Â if (total_scanned && priority < DEF_PRIORITY - 2)
> + Â Â Â Â Â Â Â Â Â Â Â congestion_wait(WRITE, HZ/10);
> +
> + Â Â Â Â Â Â Â if (sc.nr_reclaimed >= SWAP_CLUSTER_MAX)
> + Â Â Â Â Â Â Â Â Â Â Â break;
> + Â Â Â }
> +out:
> + Â Â Â if (!wmark_ok) {
> + Â Â Â Â Â Â Â cond_resched();
> +
> + Â Â Â Â Â Â Â try_to_freeze();
> +
> + Â Â Â Â Â Â Â goto loop_again;
> + Â Â Â }
> +
> + Â Â Â return sc.nr_reclaimed;
> +}
> +#else
> Âstatic unsigned long balance_mem_cgroup_pgdat(struct mem_cgroup *mem_cont,
> Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Âint order)
> Â{
> Â Â Â Âreturn 0;
> Â}
> +#endif
>
> Â/*
> Â* The background pageout daemon, started as a kernel thread
> --
> 1.7.3.1
>
>
--
Kind regards,
Minchan Kim
ÿô.nlj·ÿ±ég¬±¨ Âaþé» ®&Þ)î¦þ)íèh™¨è&£ù¢¸ÿŠæ¢ú» þÇþm§ÿÿÃÿ–)î¦þŠàb‚nö¥yÊŠ{^®wr«ë&§iÖ²('†Ûÿÿìm…éê¯Ãí¢—ÿÚ·Ÿ–ÚýiÉ¢¸ÿý½§$þŠàÿ