On 22 Jan 2024, at 22:46, Zi Yan wrote:
> From: Zi Yan <ziy@xxxxxxxxxx>
>
> Before last commit, memory compaction only migrates order-0 folios and
> skips >0 order folios. Last commit splits all >0 order folios during
> compaction. This commit migrates >0 order folios during compaction by
> keeping isolated free pages at their original size without splitting them
> into order-0 pages and using them directly during migration process.
>
> What is different from the prior implementation:
> 1. All isolated free pages are kept in a NR_PAGE_ORDERS array of page
> lists, where each page list stores free pages in the same order.
> 2. All free pages are not post_alloc_hook() processed nor buddy pages,
> although their orders are stored in first page's private like buddy
> pages.
> 3. During migration, in new page allocation time (i.e., in
> compaction_alloc()), free pages are then processed by post_alloc_hook().
> When migration fails and a new page is returned (i.e., in
> compaction_free()), free pages are restored by reversing the
> post_alloc_hook() operations using newly added
> free_pages_prepare_fpi_none().
>
> Step 3 is done for a latter optimization that splitting and/or merging free
> pages during compaction becomes easier.
>
> Note: without splitting free pages, compaction can end prematurely due to
> migration will return -ENOMEM even if there is free pages. This happens
> when no order-0 free page exist and compaction_alloc() return NULL.
>
> Signed-off-by: Zi Yan <ziy@xxxxxxxxxx>
> ---
> mm/compaction.c | 148 +++++++++++++++++++++++++++++-------------------
> mm/internal.h | 9 ++-
> mm/page_alloc.c | 6 ++
> 3 files changed, 103 insertions(+), 60 deletions(-)
>
<snip>
> @@ -1462,7 +1489,7 @@ fast_isolate_around(struct compact_control *cc, unsigned long pfn)
> if (!page)
> return;
>
> - isolate_freepages_block(cc, &start_pfn, end_pfn, &cc->freepages, 1, false);
> + isolate_freepages_block(cc, &start_pfn, end_pfn, cc->freepages, 1, false);
>
> /* Skip this pageblock in the future as it's full or nearly full */
> if (start_pfn == end_pfn && !cc->no_set_skip_hint)
> @@ -1591,7 +1618,7 @@ static void fast_isolate_freepages(struct compact_control *cc)
> nr_scanned += nr_isolated - 1;
> total_isolated += nr_isolated;
> cc->nr_freepages += nr_isolated;
> - list_add_tail(&page->lru, &cc->freepages);
> + list_add_tail(&page->lru, &cc->freepages[order].pages);
I did not increase nr_pages here, so compaction_alloc() thought no free page
was isolated.
This is the fix:
diff --git a/mm/compaction.c b/mm/compaction.c
index 335a6f6787e4..fa9993c8a389 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1638,6 +1638,7 @@ static void fast_isolate_freepages(struct compact_control *cc)
total_isolated += nr_isolated;
cc->nr_freepages += nr_isolated;
list_add_tail(&page->lru, &cc->freepages[order].pages);
+ cc->freepages[order].nr_pages++;
count_compact_events(COMPACTISOLATED, nr_isolated);
} else {
/* If isolation fails, abort the search */
I will send out v3 once I rerun vm-scalability and thpcompact.
--
Best Regards,
Yan, Zi
Attachment:
signature.asc
Description: OpenPGP digital signature
