From: Alexander Duyck <alexander.h.duyck@xxxxxxxxxxxxxxx>
Treat isolated pages as though they have already been treated. We do this
so that we can avoid trying to treat pages that have been marked for
isolation. The issue is that we don't want to run into issues where we are
treating a page, and when we put it back we find it has been moved into the
isolated migratetype, nor would we want to pull pages out of the isolated
migratetype and then find that they are now being located in a different
migratetype.
To avoid those issues we can specifically mark all isolated pages as being
"treated" and avoid special case handling for them since they will never be
merged anyway, so we can just add them to the head of the free_list.
In addition we will skip over the isolate migratetype when getting raw
pages.
Signed-off-by: Alexander Duyck <alexander.h.duyck@xxxxxxxxxxxxxxx>
---
include/linux/mmzone.h | 7 +++++++
mm/aeration.c | 8 ++++++--
mm/page_alloc.c | 2 +-
3 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index be996e8ca6b5..f749ccfcc62a 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -137,6 +137,13 @@ static inline void add_to_free_area_tail(struct page *page, struct free_area *ar
{
area->nr_free_treated++;
+#ifdef CONFIG_MEMORY_ISOLATION
+ /* Bypass membrane for isolated pages, all are considered "treated" */
+ if (migratetype == MIGRATE_ISOLATE) {
+ list_add(&page->lru, &area->free_list[migratetype]);
+ return;
+ }
+#endif
BUG_ON(area->treatment_mt != migratetype);
/* Insert page above membrane, then move membrane to the page */
diff --git a/mm/aeration.c b/mm/aeration.c
index aaf8af8d822f..f921295ed3ae 100644
--- a/mm/aeration.c
+++ b/mm/aeration.c
@@ -1,6 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/memory_aeration.h>
+#include <linux/mm.h>
#include <linux/mmzone.h>
+#include <linux/page-isolation.h>
#include <linux/gfp.h>
#include <linux/export.h>
#include <linux/delay.h>
@@ -83,8 +85,10 @@ static int __aerator_fill(struct zone *zone, unsigned int size)
* new raw pages can build. In the meantime move on
* to the next migratetype.
*/
- if (++mt >= MIGRATE_TYPES)
- mt = 0;
+ do {
+ if (++mt >= MIGRATE_TYPES)
+ mt = 0;
+ } while (is_migrate_isolate(mt));
/*
* Pull pages from free list until we have drained
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index e79c65413dc9..e3800221414b 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -989,7 +989,7 @@ static inline void __free_one_page(struct page *page,
set_page_order(page, order);
area = &zone->free_area[order];
- if (PageTreated(page)) {
+ if (is_migrate_isolate(migratetype) || PageTreated(page)) {
add_to_free_area_treated(page, area, migratetype);
return;
}
