|
Hi Tang,
On 03/16/2013 06:35 PM, Tang Chen wrote: Hi Yinghai, all, As Yinghai have implemented parsing numa info early more considerately, I think we can introduce the movablemem_map boot option again. This patch-set is based on Linux 3.9 rc-2, but need to apply Yinghai's "x86, ACPI, numa: Parse numa info early" patch-set first. Please refer to: v1: https://lkml.org/lkml/2013/3/7/642 v2: https://lkml.org/lkml/2013/3/10/47 In this part1 patch-set, we reimplemented movablemem_map boot option based on Yinghai's SRAT work. The path is like this: 1) parse SRAT, fill only existing memory into numa_meminfo, like: numa_cleanup_meminfo() { 251 const u64 low = 0; 252 const u64 high = PFN_PHYS(max_pfn); ...... 255 /* first, trim all entries */ 256 for (i = 0; i < mi->nr_blks; i++) { 257 struct numa_memblk *bi = &mi->blk[i]; 258 259 /* make sure all blocks are inside the limits */ 260 bi->start = max(bi->start, low); 261 bi->end = min(bi->end, high); 262 263 /* and there's no empty block */ 264 if (bi->start >= bi->end) 265 numa_remove_memblk_from(i--, mi); 266 } ...... } Those non-existing memory, such as memory not added yet, won't be stored in numa_meminfo. 2) initialize memory mapping for the existing memory, putting pagetables and vmemmap on local node. Since not all memory info is kept, we have to sanitize movablemem_map.map[] when we parse SRAT, so we may prevent allocating pagetables or vmemmap on local node if user specified the whole node as movable. To avoid this problem, here is my idea: 1) Store not only existing memory ranges in numa_mem_info, but all the memory info from SRAT; 2) Map only existing memory as before; 3) Do memblock limitation after memory mapping initialization using numa_meminfo, so that movablemem_map will be able to exclude pagetables and vmemmap ranges on local node. This will be done in part2 soon. How do you think? Part2 of this patch-set is under development. ======================================================================== [What we are doing] This patchset introduces a boot option for user to specify ZONE_MOVABLE memory map for each node in the system. Users can use it in two ways: 1. movablecore_map=nn[KMG]@ss[KMG] In this way, the kernel will make sure memory range from ss to ss+nn is on ZONE_MOVABLE. The hotplug info provided by SRAT will be ignored. 2. movablecore_map=acpi In this way, the kernel will use memory hotplug info in SRAT to determine ZONE_MOVABLE for each node. All the ranges user has specified will be ignored. [Why we do this] If we hot remove a memroy device, it cannot have kernel memory, because Linux cannot migrate kernel memory currently. Therefore, we have to guarantee that the hot removed memory has only movable memoroy. (Here is an exception: When we implement the node hotplug functionality, for those kernel memory whose life cycle is the same as the node, such as pagetables, vmemmap and so on, although the kernel cannot migrate them, we can still put them on local node because we can free them before we hot-remove the node. This is not implemented yet.) Linux has two boot options, kernelcore= and movablecore=, for creating movable memory. These boot options can specify the amount of memory use as kernel or movable memory. Using them, we can create ZONE_MOVABLE which has only movable memory. (NOTE: doing this will cause NUMA performance because the kernel won't be able to distribute kernel memory evenly to each node.) But it does not fulfill a requirement of memory hot remove, because even if we specify the boot options, movable memory is distributed in each node evenly. So when we want to hot remove memory which memory range is 0x80000000-0c0000000, we have no way to specify the memory as movable memory. Furthermore, even if we can use SRAT, users still need an interface to enable/disable this functionality if they don't want to lose their NUMA performance. So I think, a user interface is always needed. So we proposed this new feature which specifies memory range to use as movable memory. http://marc.info/?l=linux-mm&m=136014458829566&w=2 It seems that Mel don't like this idea.
|