> On Tue, Mar 23, 2021 at 01:04:36PM +0100, Uladzislau Rezki wrote:
> > On Mon, Mar 22, 2021 at 11:03:11PM +0000, Matthew Wilcox wrote:
> > > I suspect the vast majority of the time is spent calling alloc_pages_node()
> > > 1024 times. Have you looked at Mel's patch to do ... well, exactly what
> > > vmalloc() wants?
> > >
> > <snip>
> > - __vmalloc_node_range
> > - 45.25% __alloc_pages_nodemask
> > - 37.59% get_page_from_freelist
> [...]
> > - 44.61% 0xffffffffc047348d
> > - __vunmap
> > - 35.56% free_unref_page
>
> Hmm! I hadn't been thinking about the free side of things.
> Does this make a difference?
>
> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> index 4f5f8c907897..61d5b769fea0 100644
> --- a/mm/vmalloc.c
> +++ b/mm/vmalloc.c
> @@ -2277,16 +2277,8 @@ static void __vunmap(const void *addr, int deallocate_pages)
> vm_remove_mappings(area, deallocate_pages);
>
> if (deallocate_pages) {
> - int i;
> -
> - for (i = 0; i < area->nr_pages; i++) {
> - struct page *page = area->pages[i];
> -
> - BUG_ON(!page);
> - __free_pages(page, 0);
> - }
> + release_pages(area->pages, area->nr_pages);
> atomic_long_sub(area->nr_pages, &nr_vmalloc_pages);
> -
> kvfree(area->pages);
> }
>
Same test. 4MB allocation on a single CPU:
default: loops: 1000000 avg: 93601889 usec
patch: loops: 1000000 avg: 98217904 usec
<snip default>
- __vunmap
- 41.17% free_unref_page
- 28.42% free_pcppages_bulk
- 6.38% __mod_zone_page_state
4.79% check_preemption_disabled
2.63% __list_del_entry_valid
2.63% __list_add_valid
- 7.50% free_unref_page_commit
2.15% check_preemption_disabled
2.01% __list_add_valid
2.31% free_unref_page_prepare.part.86
0.70% free_pcp_prepare
<snip default>
<snip patch>
- __vunmap
- 45.36% release_pages
- 37.70% free_unref_page_list
- 24.70% free_pcppages_bulk
- 5.42% __mod_zone_page_state
4.23% check_preemption_disabled
2.31% __list_add_valid
2.07% __list_del_entry_valid
- 7.58% free_unref_page_commit
2.47% check_preemption_disabled
1.75% __list_add_valid
3.43% free_unref_page_prepare.part.86
- 2.39% mem_cgroup_uncharge_list
uncharge_page
<snip patch>
It is obvious that the default version is slightly better. It requires
less things to be done comparing with release_pages() variant.
>
> release_pages does a bunch of checks that are unnecessary ... we could
> probably just do:
>
> LIST_HEAD(pages_to_free);
>
> for (i = 0; i < area->nr_pages; i++) {
> struct page *page = area->pages[i];
> if (put_page_testzero(page))
> list_add(&page->lru, &pages_to_free);
> }
> free_unref_page_list(&pages_to_free);
>
> but let's see if the provided interface gets us the performance we want.
>
I will test it tomorrow. From the first glance it looks like a more light version :)
--
Vlad Rezki
