The patch titled
Subject: mm: hugetlb: add return -EAGAIN for dissolve_free_huge_page
has been removed from the -mm tree. Its filename was
mm-hugetlb-add-return-eagain-for-dissolve_free_huge_page.patch
This patch was dropped because an updated version will be merged
------------------------------------------------------
From: Muchun Song <songmuchun@xxxxxxxxxxxxx>
Subject: mm: hugetlb: add return -EAGAIN for dissolve_free_huge_page
There is a race between dissolve_free_huge_page() and put_page(),
and the race window is quite small. Theoretically, we should return
-EBUSY when we encounter this race. In fact, we have a chance to
successfully dissolve the page if we do a retry. Because the race
window is quite small. If we seize this opportunity, it is an
optimization for increasing the success rate of dissolving page.
If we free a HugeTLB page from a non-task context, it is deferred
through a workqueue. In this case, we need to flush the work.
The dissolve_free_huge_page() can be called from memory hotplug,
the caller aims to free the HugeTLB page to the buddy allocator
so that the caller can unplug the page successfully.
Link: https://lkml.kernel.org/r/20210110124017.86750-5-songmuchun@xxxxxxxxxxxxx
Signed-off-by: Muchun Song <songmuchun@xxxxxxxxxxxxx>
Cc: Andi Kleen <ak@xxxxxxxxxxxxxxx>
Cc: Michal Hocko <mhocko@xxxxxxxx>
Cc: Mike Kravetz <mike.kravetz@xxxxxxxxxx>
Cc: Naoya Horiguchi <n-horiguchi@xxxxxxxxxxxxx>
Cc: Yang Shi <shy828301@xxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
mm/hugetlb.c | 26 +++++++++++++++++++++-----
1 file changed, 21 insertions(+), 5 deletions(-)
--- a/mm/hugetlb.c~mm-hugetlb-add-return-eagain-for-dissolve_free_huge_page
+++ a/mm/hugetlb.c
@@ -1763,10 +1763,11 @@ static int free_pool_huge_page(struct hs
* nothing for in-use hugepages and non-hugepages.
* This function returns values like below:
*
- * -EBUSY: failed to dissolved free hugepages or the hugepage is in-use
- * (allocated or reserved.)
- * 0: successfully dissolved free hugepages or the page is not a
- * hugepage (considered as already dissolved)
+ * -EAGAIN: race with __free_huge_page() and can do a retry
+ * -EBUSY: failed to dissolved free hugepages or the hugepage is in-use
+ * (allocated or reserved.)
+ * 0: successfully dissolved free hugepages or the page is not a
+ * hugepage (considered as already dissolved)
*/
int dissolve_free_huge_page(struct page *page)
{
@@ -1793,8 +1794,10 @@ int dissolve_free_huge_page(struct page
* We should make sure that the page is already on the free list
* when it is dissolved.
*/
- if (unlikely(!PageHugeFreed(head)))
+ if (unlikely(!PageHugeFreed(head))) {
+ rc = -EAGAIN;
goto out;
+ }
/*
* Move PageHWPoison flag from head page to the raw error page,
@@ -1813,6 +1816,14 @@ int dissolve_free_huge_page(struct page
}
out:
spin_unlock(&hugetlb_lock);
+
+ /*
+ * If the freeing of the HugeTLB page is put on a work queue, we should
+ * flush the work before retrying.
+ */
+ if (unlikely(rc == -EAGAIN))
+ flush_work(&free_hpage_work);
+
return rc;
}
@@ -1835,7 +1846,12 @@ int dissolve_free_huge_pages(unsigned lo
for (pfn = start_pfn; pfn < end_pfn; pfn += 1 << minimum_order) {
page = pfn_to_page(pfn);
+retry:
rc = dissolve_free_huge_page(page);
+ if (rc == -EAGAIN) {
+ cpu_relax();
+ goto retry;
+ }
if (rc)
break;
}
_
Patches currently in -mm which might be from songmuchun@xxxxxxxxxxxxx are
mm-hugetlbfs-fix-cannot-migrate-the-fallocated-hugetlb-page.patch
mm-hugetlb-fix-a-race-between-freeing-and-dissolving-the-page.patch
mm-hugetlb-fix-a-race-between-isolating-and-freeing-page.patch
mm-hugetlb-remove-vm_bug_on_page-from-page_huge_active.patch
mm-memcontrol-optimize-per-lruvec-stats-counter-memory-usage.patch
mm-memcontrol-fix-nr_anon_thps-accounting-in-charge-moving.patch
mm-memcontrol-convert-nr_anon_thps-account-to-pages.patch
mm-memcontrol-convert-nr_file_thps-account-to-pages.patch
mm-memcontrol-convert-nr_shmem_thps-account-to-pages.patch
mm-memcontrol-convert-nr_shmem_pmdmapped-account-to-pages.patch
mm-memcontrol-convert-nr_file_pmdmapped-account-to-pages.patch
mm-memcontrol-make-the-slab-calculation-consistent.patch
