On Thu, Nov 19, 2020 at 7:48 AM Mike Kravetz <mike.kravetz@xxxxxxxxxx> wrote: > > On 11/13/20 2:59 AM, Muchun Song wrote: > > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > > new file mode 100644 > > index 000000000000..a6c9948302e2 > > --- /dev/null > > +++ b/mm/hugetlb_vmemmap.c > > @@ -0,0 +1,108 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > +/* > > + * Free some vmemmap pages of HugeTLB > > + * > > + * Copyright (c) 2020, Bytedance. All rights reserved. > > + * > > + * Author: Muchun Song <songmuchun@xxxxxxxxxxxxx> > > + * > > Oscar has already made some suggestions to change comments. I would suggest > changing the below text to something like the following. Thanks Mike. I will change the below comments. > > > + * Nowadays we track the status of physical page frames using struct page > > + * structures arranged in one or more arrays. And here exists one-to-one > > + * mapping between the physical page frame and the corresponding struct page > > + * structure. > > + * > > + * The HugeTLB support is built on top of multiple page size support that > > + * is provided by most modern architectures. For example, x86 CPUs normally > > + * support 4K and 2M (1G if architecturally supported) page sizes. Every > > + * HugeTLB has more than one struct page structure. The 2M HugeTLB has 512 > > + * struct page structure and 1G HugeTLB has 4096 struct page structures. But > > + * in the core of HugeTLB only uses the first 4 (Use of first 4 struct page > > + * structures comes from HUGETLB_CGROUP_MIN_ORDER.) struct page structures to > > + * store metadata associated with each HugeTLB. The rest of the struct page > > + * structures are usually read the compound_head field which are all the same > > + * value. If we can free some struct page memory to buddy system so that we > > + * can save a lot of memory. > > + * > > struct page structures (page structs) are used to describe a physical page > frame. By default, there is a one-to-one mapping from a page frame to > it's corresponding page struct. > > HugeTLB pages consist of multiple base page size pages and is supported by > many architectures. See hugetlbpage.rst in the Documentation directory for > more details. On the x86 architecture, HugeTLB pages of size 2MB and 1GB > are currently supported. Since the base page size on x86 is 4KB, a 2MB > HugeTLB page consists of 512 base pages and a 1GB HugeTLB page consists of > 4096 base pages. For each base page, there is a corresponding page struct. > > Within the HugeTLB subsystem, only the first 4 page structs are used to > contain unique information about a HugeTLB page. HUGETLB_CGROUP_MIN_ORDER > provides this upper limit. The only 'useful' information in the remaining > page structs is the compound_head field, and this field is the same for all > tail pages. > > By removing redundant page structs for HugeTLB pages, memory can returned > to the buddy allocator for other uses. > > > + * When the system boot up, every 2M HugeTLB has 512 struct page structures > > + * which size is 8 pages(sizeof(struct page) * 512 / PAGE_SIZE). > > + * > > + * HugeTLB struct pages(8 pages) page frame(8 pages) > > + * +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ > > + * | | | 0 | -------------> | 0 | > > + * | | | 1 | -------------> | 1 | > > + * | | | 2 | -------------> | 2 | > > + * | | | 3 | -------------> | 3 | > > + * | | | 4 | -------------> | 4 | > > + * | 2M | | 5 | -------------> | 5 | > > + * | | | 6 | -------------> | 6 | > > + * | | | 7 | -------------> | 7 | > > + * | | +-----------+ +-----------+ > > + * | | > > + * | | > > + * +-----------+ > > + * > > + * > > I think we want the description before the next diagram. > > Reworded description here: > > The value of compound_head is the same for all tail pages. The first page of > page structs (page 0) associated with the HugeTLB page contains the 4 page > structs necessary to describe the HugeTLB. The only use of the remaining pages > of page structs (page 1 to page 7) is to point to compound_head. Therefore, > we can remap pages 2 to 7 to page 1. Only 2 pages of page structs will be used > for each HugeTLB page. This will allow us to free the remaining 6 pages to > the buddy allocator. > > Here is how things look after remapping. > > > + * > > + * HugeTLB struct pages(8 pages) page frame(8 pages) > > + * +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ > > + * | | | 0 | -------------> | 0 | > > + * | | | 1 | -------------> | 1 | > > + * | | | 2 | -------------> +-----------+ > > + * | | | 3 | -----------------^ ^ ^ ^ ^ > > + * | | | 4 | -------------------+ | | | > > + * | 2M | | 5 | ---------------------+ | | > > + * | | | 6 | -----------------------+ | > > + * | | | 7 | -------------------------+ > > + * | | +-----------+ > > + * | | > > + * | | > > + * +-----------+ > > -- > Mike Kravetz -- Yours, Muchun
