On 2014/7/22 1:57, Nishanth Aravamudan wrote: > On 21.07.2014 [10:41:59 -0700], Tony Luck wrote: >> On Mon, Jul 21, 2014 at 10:23 AM, Nishanth Aravamudan >> <nacc@xxxxxxxxxxxxxxxxxx> wrote: >>> It seems like the issue is the order of onlining of resources on a >>> specific x86 platform? >> >> Yes. When we online a node the BIOS hits us with some ACPI hotplug events: >> >> First: Here are some new cpus > > Ok, so during this period, you might get some remote allocations. Do you > know the topology of these CPUs? That is they belong to a > (soon-to-exist) NUMA node? Can you online that currently offline NUMA > node at this point (so that NODE_DATA()) resolves, etc.)? Hi Nishanth, We have method to get the NUMA information about the CPU, and patch "[RFC Patch V1 30/30] x86, NUMA: Online node earlier when doing CPU hot-addition" tries to solve this issue by onlining NUMA node as early as possible. Actually we are trying to enable memoryless node as you have suggested. Regards! Gerry > >> Next: Here is some new memory > > And then update the NUMA topology at this point? That is, > set_cpu_numa_node/mem as appropriate so the underlying allocators do the > right thing? > >> Last; Here are some new I/O things (PCIe root ports, PCIe devices, >> IOAPICs, IOMMUs, ...) >> >> So there is a period where the node is memoryless - although that will >> generally be resolved when the memory hot plug event arrives ... that >> isn't guaranteed to occur (there might not be any memory on the node, >> or what memory there is may have failed self-test and been disabled). > > Right, but the allocator(s) generally does the right thing already in > the face of memoryless nodes -- they fallback to the nearest node. That > leads to poor performance, but is functional. Based upon the previous > thread Jiang pointed to, it seems like the real issue here isn't that > the node is memoryless, but that it's not even online yet? So NODE_DATA > access crashes? > > Thanks, > Nish > -- To unsubscribe from this list: send the line "unsubscribe linux-hotplug" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
