On 24 Jul 2024, at 20:35, Zi Yan wrote: > On 24 Jul 2024, at 18:44, Zi Yan wrote: > >> On 23 Jul 2024, at 2:41, Mike Rapoport wrote: >> >>> From: "Mike Rapoport (Microsoft)" <rppt@xxxxxxxxxx> >>> >>> Hi, >>> >>> Following the discussion about handling of CXL fixed memory windows on >>> arm64 [1] I decided to bite the bullet and move numa_memblks from x86 to >>> the generic code so they will be available on arm64/riscv and maybe on >>> loongarch sometime later. >>> >>> While it could be possible to use memblock to describe CXL memory windows, >>> it currently lacks notion of unpopulated memory ranges and numa_memblks >>> does implement this. >>> >>> Another reason to make numa_memblks generic is that both arch_numa (arm64 >>> and riscv) and loongarch use trimmed copy of x86 code although there is no >>> fundamental reason why the same code cannot be used on all these platforms. >>> Having numa_memblks in mm/ will make it's interaction with ACPI and FDT >>> more consistent and I believe will reduce maintenance burden. >>> >>> And with generic numa_memblks it is (almost) straightforward to enable NUMA >>> emulation on arm64 and riscv. >>> >>> The first 9 commits in this series are cleanups that are not strictly >>> related to numa_memblks. >>> Commits 10-16 slightly reorder code in x86 to allow extracting numa_memblks >>> and NUMA emulation to the generic code. >>> Commits 17-19 actually move the code from arch/x86/ to mm/ and commits 20-22 >>> does some aftermath cleanups. >>> Commit 23 switches arch_numa to numa_memblks. >>> Commit 24 enables usage of phys_to_target_node() and >>> memory_add_physaddr_to_nid() with numa_memblks. >>> Commit 25 moves the description for numa=fake from x86 to admin-guide >>> >>> [1] https://lore.kernel.org/all/20240529171236.32002-1-Jonathan.Cameron@xxxxxxxxxx/ >>> >>> v1: https://lore.kernel.org/all/20240716111346.3676969-1-rppt@xxxxxxxxxx >>> * add cleanup for arch_alloc_nodedata and HAVE_ARCH_NODEDATA_EXTENSION >>> * add patch that moves description of numa=fake kernel parameter from >>> x86 to admin-guide >>> * reduce rounding up of node_data allocations from PAGE_SIZE to >>> SMP_CACHE_BYTES >>> * restore single allocation attempt of numa_distance >>> * fix several comments >>> * added review tags >>> >>> Mike Rapoport (Microsoft) (25): >>> mm: move kernel/numa.c to mm/ >>> MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures >>> MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes >>> MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION >>> MIPS: loongson64: rename __node_data to node_data >>> MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION >>> mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION >>> arch, mm: move definition of node_data to generic code >>> arch, mm: pull out allocation of NODE_DATA to generic code >>> x86/numa: simplify numa_distance allocation >>> x86/numa: use get_pfn_range_for_nid to verify that node spans memory >>> x86/numa: move FAKE_NODE_* defines to numa_emu >>> x86/numa_emu: simplify allocation of phys_dist >>> x86/numa_emu: split __apicid_to_node update to a helper function >>> x86/numa_emu: use a helper function to get MAX_DMA32_PFN >>> x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned >>> mm: introduce numa_memblks >>> mm: move numa_distance and related code from x86 to numa_memblks >>> mm: introduce numa_emulation >>> mm: numa_memblks: introduce numa_memblks_init >>> mm: numa_memblks: make several functions and variables static >>> mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing >>> meminfo >>> arch_numa: switch over to numa_memblks >>> mm: make range-to-target_node lookup facility a part of numa_memblks >>> docs: move numa=fake description to kernel-parameters.txt >>> >> Hi, >> >> I have tested this series on both x86_64 and arm64. It works fine on x86_64. >> All numa=fake= options work as they did before the series. >> >> But I am not able to boot the kernel (no printout at all) on arm64 VM >> (Mac mini M1 VMWare). By git bisecting, arch_numa: switch over to numa_memblks >> is the first patch causing the boot failure. I see the warning: >> >> WARNING: modpost: vmlinux: section mismatch in reference: numa_add_cpu+0x1c (section: .text) -> early_cpu_to_node (section: .init.text) >> >> I am not sure if it is red herring or not, since changing early_cpu_to_node >> to cpu_to_node in numa_add_cpu() from mm/numa_emulation.c did get rid of the >> warning, but the system still failed to boot. >> >> Please note that you need binutils 2.40 to build the arm64 kernel, since there >> is a bug(https://sourceware.org/bugzilla/show_bug.cgi?id=31924) in 2.42 preventing >> arm64 kernel from booting as well. >> >> My config is attached. > > I get more info after adding earlycon to the boot option. > pgdat is NULL, causing issues when free_area_init_node() is dereferencing > it at first WARN_ON. > > FYI, my build is this series on top of v6.10 instead of the base commit, > where the series applies cleanly on top v6.10. OK, the issue comes from that my arm64 VM has no ACPI but x86_64 VM has it, thus on arm64 VM numa_init(arch_acpi_numa_ini) failed in arch_numa_init() and the code falls back to numa_init(dummy_numa_init). In dummy_numa_init(), before patch 23 "arch_numa: switch over to numa_memblks", numa_add_memblk() from drivers/base/arch_numa.c is called on arm64, which unconditionally set 0 to numa_nodes_parsed. This is missing in the x86 version of numa_add_memblk(), which is now used by all arch. By adding the patch below, my arm64 kernel boots in the VM. diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c index 806550239d08..354f15b8d9b7 100644 --- a/drivers/base/arch_numa.c +++ b/drivers/base/arch_numa.c @@ -279,6 +279,7 @@ static int __init dummy_numa_init(void) pr_err("NUMA init failed\n"); return ret; } + node_set(0, numa_nodes_parsed); numa_off = true; return 0; Feel free to add Tested-by: Zi Yan <ziy@xxxxxxxxxx> # for x86_64 and arm64 after you incorporate the fix. -- Best Regards, Yan, Zi
Attachment:
signature.asc
Description: OpenPGP digital signature
