On Thu 26-07-18 15:12:42, Michal Hocko wrote: > On Thu 26-07-18 21:09:04, Baoquan He wrote: > > On 07/26/18 at 02:59pm, Michal Hocko wrote: > > > On Wed 25-07-18 14:48:13, Baoquan He wrote: > > > > On 07/23/18 at 04:34pm, Michal Hocko wrote: > > > > > On Thu 19-07-18 23:17:53, Baoquan He wrote: > > > > > > Kexec has been a formal feature in our distro, and customers owning > > > > > > those kind of very large machine can make use of this feature to speed > > > > > > up the reboot process. On uefi machine, the kexec_file loading will > > > > > > search place to put kernel under 4G from top to down. As we know, the > > > > > > 1st 4G space is DMA32 ZONE, dma, pci mmcfg, bios etc all try to consume > > > > > > it. It may have possibility to not be able to find a usable space for > > > > > > kernel/initrd. From the top down of the whole memory space, we don't > > > > > > have this worry. > > > > > > > > > > I do not have the full context here but let me note that you should be > > > > > careful when doing top-down reservation because you can easily get into > > > > > hotplugable memory and break the hotremove usecase. We even warn when > > > > > this is done. See memblock_find_in_range_node > > > > > > > > Kexec read kernel/initrd file into buffer, just search usable positions > > > > for them to do the later copying. You can see below struct kexec_segment, > > > > for the old kexec_load, kernel/initrd are read into user space buffer, > > > > the @buf stores the user space buffer address, @mem stores the position > > > > where kernel/initrd will be put. In kernel, it calls > > > > kimage_load_normal_segment() to copy user space buffer to intermediate > > > > pages which are allocated with flag GFP_KERNEL. These intermediate pages > > > > are recorded as entries, later when user execute "kexec -e" to trigger > > > > kexec jumping, it will do the final copying from the intermediate pages > > > > to the real destination pages which @mem pointed. Because we can't touch > > > > the existed data in 1st kernel when do kexec kernel loading. With my > > > > understanding, GFP_KERNEL will make those intermediate pages be > > > > allocated inside immovable area, it won't impact hotplugging. But the > > > > @mem we searched in the whole system RAM might be lost along with > > > > hotplug. Hence we need do kexec kernel again when hotplug event is > > > > detected. > > > > > > I am not sure I am following. If @mem is placed at movable node then the > > > memory hotremove simply won't work, because we are seeing reserved pages > > > and do not know what to do about them. They are not migrateable. > > > Allocating intermediate pages from other nodes doesn't really help. > > > > OK, I forgot the 2nd kernel which kexec jump into. It won't impact hotremove > > in 1st kernel, it does impact the kernel which kexec jump into if kernel > > is at top of system RAM and the top RAM is in movable node. > > It will affect the 1st kernel (which does the memblock allocation > top-down) as well. For reasons mentioned above. And btw. in the ideal world, we would restrict the memblock allocation top-down from the non-movable nodes. But I do not think we have that information ready at the time when the reservation is done. -- Michal Hocko SUSE Labs
