The patch titled
Subject: mm: speedup in __early_pfn_to_nid
has been added to the -mm tree. Its filename is
mm-speedup-in-__early_pfn_to_nid.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/SubmitChecklist when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Russ Anderson <rja@xxxxxxx>
Subject: mm: speedup in __early_pfn_to_nid
When booting on a large memory system, the kernel spends considerable time
in memmap_init_zone() setting up memory zones. Analysis shows significant
time spent in __early_pfn_to_nid().
The routine memmap_init_zone() checks each PFN to verify the nid is valid.
__early_pfn_to_nid() sequentially scans the list of pfn ranges to find
the right range and returns the nid. This does not scale well. On a 4 TB
(single rack) system there are 308 memory ranges to scan. The higher the
PFN the more time spent sequentially spinning through memory ranges.
Since memmap_init_zone() increments pfn, it will almost always be looking
for the same range as the previous pfn, so check that range first. If it
is in the same range, return that nid. If not, scan the list as before.
A 4 TB (single rack) UV1 system takes 512 seconds to get through the zone
code. This performance optimization reduces the time by 189 seconds, a
36% improvement.
A 2 TB (single rack) UV2 system goes from 212.7 seconds to 99.8 seconds, a
112.9 second (53%) reduction.
Signed-off-by: Russ Anderson <rja@xxxxxxx>
Cc: David Rientjes <rientjes@xxxxxxxxxx>
Cc: Ingo Molnar <mingo@xxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Cc: "H. Peter Anvin" <hpa@xxxxxxxxx>
Cc: "Luck, Tony" <tony.luck@xxxxxxxxx>
Cc: Yinghai Lu <yinghai@xxxxxxxxxx>
Cc: Lin Feng <linfeng@xxxxxxxxxxxxxx>
Cc: KOSAKI Motohiro <kosaki.motohiro@xxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
arch/ia64/mm/numa.c | 15 ++++++++++++++-
mm/page_alloc.c | 15 ++++++++++++++-
2 files changed, 28 insertions(+), 2 deletions(-)
diff -puN arch/ia64/mm/numa.c~mm-speedup-in-__early_pfn_to_nid arch/ia64/mm/numa.c
--- a/arch/ia64/mm/numa.c~mm-speedup-in-__early_pfn_to_nid
+++ a/arch/ia64/mm/numa.c
@@ -61,13 +61,26 @@ paddr_to_nid(unsigned long paddr)
int __meminit __early_pfn_to_nid(unsigned long pfn)
{
int i, section = pfn >> PFN_SECTION_SHIFT, ssec, esec;
+ /*
+ NOTE: The following SMP-unsafe globals are only used early
+ in boot when the kernel is running single-threaded.
+ */
+ static unsigned long last_start_pfn, last_end_pfn;
+ static int last_nid;
+
+ if (section >= last_ssec && section < last_esec)
+ return last_nid;
for (i = 0; i < num_node_memblks; i++) {
ssec = node_memblk[i].start_paddr >> PA_SECTION_SHIFT;
esec = (node_memblk[i].start_paddr + node_memblk[i].size +
((1L << PA_SECTION_SHIFT) - 1)) >> PA_SECTION_SHIFT;
- if (section >= ssec && section < esec)
+ if (section >= ssec && section < esec) {
+ last_ssec = ssec;
+ last_esec = esec;
+ last_nid = node_memblk[i].nid
return node_memblk[i].nid;
+ }
}
return -1;
diff -puN mm/page_alloc.c~mm-speedup-in-__early_pfn_to_nid mm/page_alloc.c
--- a/mm/page_alloc.c~mm-speedup-in-__early_pfn_to_nid
+++ a/mm/page_alloc.c
@@ -4186,10 +4186,23 @@ int __meminit __early_pfn_to_nid(unsigne
{
unsigned long start_pfn, end_pfn;
int i, nid;
+ /*
+ NOTE: The following SMP-unsafe globals are only used early
+ in boot when the kernel is running single-threaded.
+ */
+ static unsigned long last_start_pfn, last_end_pfn;
+ static int last_nid;
+
+ if (last_start_pfn <= pfn && pfn < last_end_pfn)
+ return last_nid;
for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid)
- if (start_pfn <= pfn && pfn < end_pfn)
+ if (start_pfn <= pfn && pfn < end_pfn) {
+ last_start_pfn = start_pfn;
+ last_end_pfn = end_pfn;
+ last_nid = nid;
return nid;
+ }
/* This is a memory hole */
return -1;
}
_
Patches currently in -mm which might be from rja@xxxxxxx are
mm-speedup-in-__early_pfn_to_nid.patch
mm-speedup-in-__early_pfn_to_nid-fix.patch
mm-speedup-in-__early_pfn_to_nid-fix-fix-2.patch
--
To unsubscribe from this list: send the line "unsubscribe mm-commits" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
