The patch titled
Subject: hugetlb: optimize demote vmemmap optimizatized pages
has been added to the -mm tree. Its filename is
hugetlb-optimized-demote-vmemmap-optimizatized-pages.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/hugetlb-optimized-demote-vmemmap-optimizatized-pages.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/hugetlb-optimized-demote-vmemmap-optimizatized-pages.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Mike Kravetz <mike.kravetz@xxxxxxxxxx>
Subject: hugetlb: optimize demote vmemmap optimizatized pages
Put all the pieces together to optimize the process of demoting vmemmap
optimized pages.
Instead of allocating all vmemmap pages for a page to be demoted, use the
demote_huge_page_vmemmap routine which will only allocate/map pages needed
for the demoted pages.
For vmemmap optimized pages, use the destroy_compound_gigantic_page and
prep_compound_gigantic_page routines during demote. These routines can
deal with vmemmap optimized pages, and know which page structs are
writable.
Link: https://lkml.kernel.org/r/20210816224953.157796-9-mike.kravetz@xxxxxxxxxx
Signed-off-by: Mike Kravetz <mike.kravetz@xxxxxxxxxx>
Cc: David Hildenbrand <david@xxxxxxxxxx>
Cc: David Rientjes <rientjes@xxxxxxxxxx>
Cc: Michal Hocko <mhocko@xxxxxxxx>
Cc: Muchun Song <songmuchun@xxxxxxxxxxxxx>
Cc: Naoya Horiguchi <naoya.horiguchi@xxxxxxxxx>
Cc: Oscar Salvador <osalvador@xxxxxxx>
Cc: Zi Yan <ziy@xxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
mm/hugetlb.c | 27 ++++++++++++++++++++++++---
1 file changed, 24 insertions(+), 3 deletions(-)
--- a/mm/hugetlb.c~hugetlb-optimized-demote-vmemmap-optimizatized-pages
+++ a/mm/hugetlb.c
@@ -3310,13 +3310,14 @@ static int demote_free_huge_page(struct
int i, nid = page_to_nid(page);
struct hstate *target_hstate;
bool cma_page = HPageCma(page);
+ bool vmemmap_optimized = HPageVmemmapOptimized(page);
target_hstate = size_to_hstate(PAGE_SIZE << h->demote_order);
remove_hugetlb_page_for_demote(h, page, false);
spin_unlock_irq(&hugetlb_lock);
- if (alloc_huge_page_vmemmap(h, page)) {
+ if (demote_huge_page_vmemmap(h, page)) {
/* Allocation of vmemmmap failed, we can not demote page */
spin_lock_irq(&hugetlb_lock);
set_page_refcounted(page);
@@ -3328,16 +3329,36 @@ static int demote_free_huge_page(struct
* Use destroy_compound_gigantic_page_for_demote for all huge page
* sizes as it will not ref count pages.
*/
- destroy_compound_gigantic_page_for_demote(page, huge_page_order(h));
+ if (vmemmap_optimized)
+ /*
+ * If page is vmemmmap optimized, then demote_huge_page_vmemmap
+ * added vmammap for each smaller page of target order size.
+ * We must update/destroy all each of these smaller pages.
+ */
+ for (i = 0; i < pages_per_huge_page(h);
+ i += pages_per_huge_page(target_hstate))
+ destroy_compound_gigantic_page_for_demote(page + i,
+ huge_page_order(target_hstate));
+ else
+ destroy_compound_gigantic_page_for_demote(page,
+ huge_page_order(h));
for (i = 0; i < pages_per_huge_page(h);
i += pages_per_huge_page(target_hstate)) {
- if (hstate_is_gigantic(target_hstate))
+ /*
+ * Use gigantic page prep for vmemmap_optimized pages of
+ * all sizes as it has special vmemmap logic. The generic
+ * prep routine does not and should not know about hugetlb
+ * vmemmap optimizations.
+ */
+ if (hstate_is_gigantic(target_hstate) || vmemmap_optimized)
prep_compound_gigantic_page_for_demote(page + i,
target_hstate->order);
else
prep_compound_page(page + i, target_hstate->order);
set_page_private(page + i, 0);
+ if (vmemmap_optimized)
+ SetHPageVmemmapOptimized(page + i);
set_page_refcounted(page + i);
prep_new_huge_page(target_hstate, page + i, nid);
if (cma_page)
_
Patches currently in -mm which might be from mike.kravetz@xxxxxxxxxx are
hugetlb-simplify-prep_compound_gigantic_page-ref-count-racing-code.patch
hugetlb-drop-ref-count-earlier-after-page-allocation.patch
hugetlb-before-freeing-hugetlb-page-set-dtor-to-appropriate-value.patch
hugetlb-add-demote-hugetlb-page-sysfs-interfaces.patch
hugetlb-add-hpagecma-flag-and-code-to-free-non-gigantic-pages-in-cma.patch
hugetlb-add-demote-bool-to-gigantic-page-routines.patch
hugetlb-add-hugetlb-demote-page-support.patch
hugetlb-document-the-demote-sysfs-interfaces.patch
hugetlb-vmemmap-optimizations-when-demoting-hugetlb-pages.patch
hugetlb-prepare-destroy-and-prep-routines-for-vmemmap-optimized-pages.patch
hugetlb-optimized-demote-vmemmap-optimizatized-pages.patch
