On 03/18/20 at 02:58pm, Vitaly Kuznetsov wrote: > Baoquan He <bhe@xxxxxxxxxx> writes: > > > On 03/17/20 at 11:49am, David Hildenbrand wrote: > >> Distributions nowadays use udev rules ([1] [2]) to specify if and > >> how to online hotplugged memory. The rules seem to get more complex with > >> many special cases. Due to the various special cases, > >> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE cannot be used. All memory hotplug > >> is handled via udev rules. > >> > >> Everytime we hotplug memory, the udev rule will come to the same > >> conclusion. Especially Hyper-V (but also soon virtio-mem) add a lot of > >> memory in separate memory blocks and wait for memory to get onlined by user > >> space before continuing to add more memory blocks (to not add memory faster > >> than it is getting onlined). This of course slows down the whole memory > >> hotplug process. > >> > >> To make the job of distributions easier and to avoid udev rules that get > >> more and more complicated, let's extend the mechanism provided by > >> - /sys/devices/system/memory/auto_online_blocks > >> - "memhp_default_state=" on the kernel cmdline > >> to be able to specify also "online_movable" as well as "online_kernel" > > > > This patch series looks good, thanks. Since Andrew has merged it to -mm again, > > I won't add my Reviewed-by to bother. > > > > Hi David, Vitaly > > > > There are several things unclear to me. > > > > So, these improved interfaces are used to alleviate the burden of the > > existing udev rules, or try to replace it? As you know, we have been > > using udev rules to interact between kernel and user space on bare metal, > > and guests who want to hot add/remove. > > With 'auto_online_blocks' interface you don't need the udev rule. David > is trying to make it more versatile. > > > > > And also the OOM issue in hyperV when onlining pages after adding memory > > block. I am not a virt devel expert, could this happen on bare metal > > system? > > Yes - in theory, very unlikely - in practice. > > The root cause of the problem here is adding more memory to the system > requires memory (page tables, memmaps,..) so if your system is low on > memory and you're trying to hotplug A LOT you may run into OOM before > you're able to online anything. With bare metal it's usualy not the > case: servers, which are able to hotplug memory, are usually booted with > enough memory and memory hotplug is a manual action (you need to insert > DIMMs!). But, if you boot your server with e.g. 4G, almost exhaust it > and then try to hotplug e.g. 256G ... well, OOM is almost guaranteed. Thanks for this detailed explanation. I finally know why this is a problem in hyperV. But with the current mechanism, it will happen on any system if thing is done like this. Is there a reason hyperV need boot with small memory, then enlarge it with huge memory? Since it's a real case in hyperV, I guess there must be reason, I am just curious. > With virtual machines it's very common (e.g. with Hyper-V VMs) to boot > them with low memory and hotplug it (automatically, by some management > software) when neededm thus the problem is way more common.
