On 17.09.23 12:46, Maciej S. Szmigiero wrote:
On 8.09.2023 16:21, David Hildenbrand wrote:We want to support memory devices that can automatically decide how many memslots they will use. In the worst case, they have to use a single memslot. The target use cases are virtio-mem and the hyper-v balloon. Let's calculate a reasonable limit such a memory device may use, and instruct the device to make a decision based on that limit. Use a simple heuristic that considers: * A memslot soft-limit for all memory devices of 256; also, to not consume too many memslots -- which could harm performance. * Actually still free and unreserved memslots * The percentage of the remaining device memory region that memory device will occupy. Further, while we properly check before plugging a memory device whether there still is are free memslots, we have other memslot consumers (such as boot memory, PCI BARs) that don't perform any checks and might dynamically consume memslots without any prior reservation. So we might succeed in plugging a memory device, but once we dynamically map a PCI BAR we would be in trouble. Doing accounting / reservation / checks for all such users is problematic (e.g., sometimes we might temporarily split boot memory into two memslots, triggered by the BIOS). We use the historic magic memslot number of 509 as orientation to when supporting 256 memory devices -> memslots (leaving 253 for boot memory and other devices) has been proven to work reliable. We'll fallback to suggesting a single memslot if we don't have at least 509 total memslots. Plugging vhost devices with less than 509 memslots available while we have memory devices plugged that consume multiple memslots due to automatic decisions can be problematic. Most configurations might just fail due to "limit < used + reserved", however, it can also happen that these memory devices would suddenly consume memslots that would actually be required by other memslot consumers (boot, PCI BARs) later. Note that this has always been sketchy with vhost devices that support only a small number of memslots; but we don't want to make it any worse.So let's keep it simple and simply reject plugging such vhost devices in such a configuration. Eventually, all vhost devices that want to be fully compatible with such memory devices should support a decent number of memslots (>= 509). Signed-off-by: David Hildenbrand <david@xxxxxxxxxx> ---Reviewed-by: Maciej S. Szmigiero <maciej.szmigiero@xxxxxxxxxx>
Thanks!
I would be nice to ultimately allow raising the 509 memslot limit, considering that KVM had supported 32k memslots for more than two years now and had a much more scalable implementation since early 2022.
It's all tricky due to vhost (and hotplug of such devices) and the QEMU internal address translation (which isn't that scalable).
I was thinking about having a parameter to configure the number of memslots for memory devices, such that one could manually raise the "256" limit for memory devices.
But for now I kept it simple, because it all turned out to become way to complicated.
-- Cheers, David / dhildenb
