On Tue, Jun 01, 2021 at 12:48:15PM +0100, Matthew Wilcox wrote:
> On Tue, Jun 01, 2021 at 01:31:43PM +0800, Huang Ying wrote:
> > With commit 09854ba94c6a ("mm: do_wp_page() simplification"), after
> > COW, the idle swap cache page (neither the page nor the corresponding
> > swap entry is mapped by any process) will be left in the LRU list,
> > even if it's in the active list or the head of the inactive list. So,
> > the page reclaimer may take quite some overhead to reclaim these
> > actually unused pages.
> >
> > To help the page reclaiming, in this patch, after COW, the idle swap
> > cache page will be tried to be freed. To avoid to introduce much
> > overhead to the hot COW code path,
> >
> > a) there's almost zero overhead for non-swap case via checking
> > PageSwapCache() firstly.
> >
> > b) the page lock is acquired via trylock only.
> >
> > To test the patch, we used pmbench memory accessing benchmark with
> > working-set larger than available memory on a 2-socket Intel server
> > with a NVMe SSD as swap device. Test results shows that the pmbench
> > score increases up to 23.8% with the decreased size of swap cache and
> > swapin throughput.
>
> So 2 percentage points better than my original idea? Sweet.
>
> > diff --git a/mm/memory.c b/mm/memory.c
> > index 2b7ffcbca175..d44425820240 100644
> > --- a/mm/memory.c
> > +++ b/mm/memory.c
> > @@ -3104,6 +3104,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
> > munlock_vma_page(old_page);
> > unlock_page(old_page);
> > }
> > + if (page_copied)
> > + free_swap_cache(old_page);
> > put_page(old_page);
> > }
> > return page_copied ? VM_FAULT_WRITE : 0;
>
> Why not ...
>
> if (page_copied)
> free_page_and_swap_cache(old_page);
> else
> put_page(old_page);
>
> then you don't need to expose free_swap_cache(). Or does the test for
> huge_zero_page mess this up?
It's free_page[s]_and_swap_cache() we should reconsider, IMO.
free_swap_cache() makes for a clean API function that does one thing,
and does it right. free_page_and_swap_cache() combines two independent
operations, which has the habit of accumulating special case-handling
for some callers that is unncessary overhead for others (Abstraction
Inversion anti-pattern).
For example, free_page_and_swap_cache() adds an is_huge_zero_page()
check around the put_page() for the tlb batching code. This isn't
needed here. AFAICS it is also unnecessary for the other callsite,
__collapse_huge_page_copy(), where context rules out zero pages.
The common put_page() in Huang's version also makes it slighly easier
to follow the lifetime of old_page.
So I'd say exposing free_swap_cache() is a good move, for this patch
and in general.
