> > 2. Coarse grained memory increases for 'normal' memory. > > Can use memory hot-plug. Recovery of capacity likely to only be possible on > > VM shutdown. > > Is there are reason "movable" (ZONE_MOVABLE) is not an option, at least in > some setups? If not, why? > This seems like a bit of a muddied conversation. "'normal' memory" has no defined meaning - so lets clear this up a bit There is: * System-RAM (memory managed by kernel allocators) * Special Purpose Memory (generally presented as DAX) System-RAM is managed as zones - the relevant ones are * ZONE_NORMAL allows both movable and non-movable allocations * ZONE_MOVABLE only allows non-movable allocations (Caveat: this generally only applies to allocation, you can violate this with stuff like pinning) Hotplug can be thought of as two discrete mechanisms * Exposing capacity to the kernel (CXL DCD Transactions) * Exposing capacity to allocators (mm/memory-hotplug.c) 1) if the intent is to primarily utilize dynamic capacity for VMs, then the host does not need (read: should not need) to map the memory as System-RAM in the host. The VMM should be made to consume it directly via DAX or otherwise. That capacity is almost by definition "Capital G Guaranteed" to be reclaimable regardless of what the guest does. A VMM can force a guest to let go of resources - that's its job. 2) if the intent is to provide dynamic capacity to a host as System-RAM, then recoverability is dictated by system usage of that capacity. If onlined into ZONE_MOVABLE, then if the system has avoided doing things like pinning those pages it should *generally* be recoverable (but not guaranteed). For the virtualization discussion: Hotplug and recoverability is a non-issue. The capacity should never be exposed to system allocators and the VMM should be made to consume special purpose memory directly. That's on the VMM/orchestration software to get right. For the host System-RAM discussion: Auto-onlined hotplug capacity presently defaults to ZONE_NORMAL, but we discussed (yesterday, at Plumbers) changing this default to ZONE_MOVABLE. The only concern is when insufficient ZONE_NORMAL exists to support ZONE_MOVABLE capacity - but this is unlikely to be the general scenario AND can be mitigated w/ existing mechanisms. Manually onlined capacity defaults to ZONE_MOVABLE. It would be nice to make this behavior consistent, since the general opinion appears to be that this capacity should default to ZONE_MOVABLE. ~Gregory
