On Fri, Aug 21, 2015 at 11:19 AM, Luck, Tony <tony.luck@xxxxxxxxx> wrote: > On Tue, Nov 04, 2014 at 04:29:44PM +0800, Daniel J Blueman wrote: >> On large-memory x86-64 systems of 64GB or more with memory hot-plug >> enabled, use a 2GB memory block size. Eg with 64GB memory, this reduces >> the number of directories in /sys/devices/system/memory from 512 to 32, >> making it more manageable, and reducing the creation time accordingly. >> >> This caveat is that the memory can't be offlined (for hotplug or otherwise) >> with finer 128MB granularity, but this is unimportant due to the high >> memory densities generally used with such large-memory systems, where >> eg a single DIMM is the order of 16GB. > > git bisect points to this commit as the cause of a panic on my > machine: > > [ 4.518415] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 > [ 4.525882] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000) > [ 4.536280] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820 > [ 4.544344] PCI: Using configuration type 1 for base access > [ 4.550778] BUG: unable to handle kernel paging request at ffffea0078000020 > [ 4.558572] IP: [<ffffffff8142ab0d>] register_mem_sect_under_ ... > so the older code will look at max_pfn and set memory block size: > > [ 3.021752] memory block size : 256MB > > I think the problem is more connected to the strange max_pfn rather > than the holes ... but will defer to wiser heads. > > If the problem is with max_pfn ... I don't think it is a safe assumption > that systems with >64GB memory will have 2GB aligned max_pfn. That commit could be reverted. According to https://lkml.org/lkml/2014/11/10/123 I had attached patch for my test setups for a while. Yinghai
Subject: [PATCH] x86, mm: put memory block size probing back commit bdee237c0343 ("x86: mm: Use 2GB memory block size on large-memory x86-64 systems") let system with more than 64GiB ram just use 2G as memory block size without probing. found one system: has memory map like: [0x00000000-0x60000000) [0x100000000-0x20a0000000) We should use 0x20000000 in this case. So can not assume system with big memory have 2g tail anymore. So revert it to put probing back. Fixes: bdee237c0343 ("x86: mm: Use 2GB memory block size on large-memory x86-64 systems") Signed-off-by: Yinghai Lu <yinghai@xxxxxxxxxx> --- arch/x86/mm/init_64.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) Index: linux-2.6/arch/x86/mm/init_64.c =================================================================== --- linux-2.6.orig/arch/x86/mm/init_64.c +++ linux-2.6/arch/x86/mm/init_64.c @@ -52,6 +52,7 @@ #include <asm/numa.h> #include <asm/cacheflush.h> #include <asm/init.h> +#include <asm/uv/uv.h> #include <asm/setup.h> #include "mm_internal.h" @@ -1204,10 +1205,12 @@ static unsigned long probe_memory_block_ /* start from 2g */ unsigned long bz = 1UL<<31; - if (totalram_pages >= (64ULL << (30 - PAGE_SHIFT))) { - pr_info("Using 2GB memory block size for large-memory system\n"); +#ifdef CONFIG_X86_UV + if (is_uv_system()) { + printk(KERN_INFO "UV: memory block size 2GB\n"); return 2UL * 1024 * 1024 * 1024; } +#endif /* less than 64g installed */ if ((max_pfn << PAGE_SHIFT) < (16UL << 32))