On Sun, Sep 18, 2022 at 2:01 AM Yu Zhao <yuzhao@xxxxxxxxxx> wrote:
...
> This patch uses the following optimizations when walking page tables:
> 1. It tracks the usage of mm_struct's between context switches so that
> page table walkers can skip processes that have been sleeping since
> the last iteration.
...
> @@ -672,6 +672,22 @@ struct mm_struct {
> */
> unsigned long ksm_merging_pages;
> #endif
> +#ifdef CONFIG_LRU_GEN
> + struct {
> + /* this mm_struct is on lru_gen_mm_list */
> + struct list_head list;
> + /*
> + * Set when switching to this mm_struct, as a hint of
> + * whether it has been used since the last time per-node
> + * page table walkers cleared the corresponding bits.
> + */
> + unsigned long bitmap;
...
> +static inline void lru_gen_use_mm(struct mm_struct *mm)
> +{
> + /*
> + * When the bitmap is set, page reclaim knows this mm_struct has been
> + * used since the last time it cleared the bitmap. So it might be worth
> + * walking the page tables of this mm_struct to clear the accessed bit.
> + */
> + WRITE_ONCE(mm->lru_gen.bitmap, -1);
> +}
...
> @@ -5180,6 +5180,7 @@ context_switch(struct rq *rq, struct task_struct *prev,
> * finish_task_switch()'s mmdrop().
> */
> switch_mm_irqs_off(prev->active_mm, next->mm, next);
> + lru_gen_use_mm(next->mm);
>
> if (!prev->mm) { // from kernel
> /* will mmdrop() in finish_task_switch(). */
Adding Ingo, Peter, Juri and Vincent for the bit above, per previous
discussion here:
https://lore.kernel.org/r/CAOUHufY91Eju-g1+xbUsGkGZ-cwBm78v+S_Air7Cp8mAnYJVYA@xxxxxxxxxxxxxx/
I trimmed 99% of this patch to save your time. In case you want to
hear the whole story:
https://lore.kernel.org/r/20220918080010.2920238-9-yuzhao@xxxxxxxxxx/
