From: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx>
Add another layer of fallback policy to make the home node concept
useful from a memory allocation PoV.
This changes the mpol order to:
- vma->vm_ops->get_policy [if applicable]
- vma->vm_policy [if applicable]
- task->mempolicy
- tsk_home_node() preferred [NEW]
- default_policy
Note that the tsk_home_node() policy has Migrate-on-Fault enabled to
facilitate efficient on-demand memory migration.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@xxxxxxxxx>
Cc: Paul Turner <pjt@xxxxxxxxxx>
Cc: Lee Schermerhorn <Lee.Schermerhorn@xxxxxx>
Cc: Christoph Lameter <cl@xxxxxxxxx>
Cc: Rik van Riel <riel@xxxxxxxxxx>
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Signed-off-by: Ingo Molnar <mingo@xxxxxxxxxx>
Signed-off-by: Mel Gorman <mgorman@xxxxxxx>
---
include/uapi/linux/mempolicy.h | 9 ++++++++-
mm/mempolicy.c | 30 ++++++++++++++++++++----------
2 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/include/uapi/linux/mempolicy.h b/include/uapi/linux/mempolicy.h
index b25064f..bc7b611 100644
--- a/include/uapi/linux/mempolicy.h
+++ b/include/uapi/linux/mempolicy.h
@@ -69,7 +69,14 @@ enum mpol_rebind_step {
#define MPOL_F_LOCAL (1 << 1) /* preferred local allocation */
#define MPOL_F_REBINDING (1 << 2) /* identify policies in rebinding */
#define MPOL_F_MOF (1 << 3) /* this policy wants migrate on fault */
-#define MPOL_F_MORON (1 << 4) /* Migrate On pte_numa Reference On Node */
+#define MPOL_F_HOME (1 << 4) /*
+ * Migrate towards referencing node.
+ * By building up stats on faults, the
+ * scheduler will reinforce the choice
+ * by identifying a home node and
+ * queueing the task on that node
+ * where possible.
+ */
#endif /* _UAPI_LINUX_MEMPOLICY_H */
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index f2111b7..076f8f8 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -126,9 +126,10 @@ static struct mempolicy *get_task_policy(struct task_struct *p)
int node;
if (!pol) {
- node = numa_node_id();
- if (node != -1)
- pol = &preferred_node_policy[node];
+ node = tsk_home_node(p);
+ if (node == -1)
+ node = numa_node_id();
+ pol = &preferred_node_policy[node];
/* preferred_node_policy is not initialised early in boot */
if (!pol->mode)
@@ -2422,12 +2423,21 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long
BUG();
}
- /*
- * Moronic node selection policy. Migrate the page to the node that is
- * currently referencing it
- */
- if (pol->flags & MPOL_F_MORON)
- polnid = numa_node_id();
+ /* Migrate pages towards their home node or the referencing node */
+ if (pol->flags & MPOL_F_HOME) {
+ /*
+ * Make a placement decision based on the home node.
+ * NOTE: Potentially this can result in a remote->remote
+ * copy but it's not migrated now the numa_fault will
+ * be lost or accounted for incorrectly making it a rock
+ * and a hard place.
+ */
+ polnid = tsk_home_node(current);
+ if (polnid == -1) {
+ /* No home node, migrate to the referencing node */
+ polnid = numa_node_id();
+ }
+ }
if (curnid != polnid)
ret = polnid;
@@ -2621,7 +2631,7 @@ void __init numa_policy_init(void)
preferred_node_policy[nid] = (struct mempolicy) {
.refcnt = ATOMIC_INIT(1),
.mode = MPOL_PREFERRED,
- .flags = MPOL_F_MOF | MPOL_F_MORON,
+ .flags = MPOL_F_MOF | MPOL_F_HOME,
.v = { .preferred_node = nid, },
};
}
--
1.7.9.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>