Subject: [RFC] Memory controller exploitation in libvirt
Memory CGroup is a kernel feature that can be exploited effectively in the
current libvirt/qemu driver. Here is a shot at that.
At present, QEmu uses memory ballooning feature, where the memory can be
inflated/deflated as and when needed, co-operatively between the host and
the guest. There should be some mechanism where the host can have more
control over the guests memory usage. Memory CGroup provides features such
as hard-limit and soft-limit for memory, and hard-limit for swap area.
Design 1: Provide new API and XML changes for resource management
=================================================================
All the memory controller tunables are not supported with the current
abstractions provided by the libvirt API. libvirt works on various OS. This
new API will support GNU/Linux initially and as and when other platforms
starts supporting memory tunables, the interface could be enabled for
them. Adding following two function pointer to the virDriver interface.
1) domainSetMemoryParameters: which would take one or more name-value
pairs. This makes the API extensible, and agnostic to the kind of
parameters supported by various Hypervisors.
2) domainGetMemoryParameters: For getting current memory parameters
Corresponding libvirt public API:
int virDomainSetMemoryParamters (virDomainPtr domain,
virMemoryParamterPtr params,
unsigned int nparams);
int virDomainGetMemoryParamters (virDomainPtr domain,
virMemoryParamterPtr params,
unsigned int nparams);
Parameter list supported:
MemoryHardLimits (memory.limits_in_bytes) - Maximum memory
MemorySoftLimits (memory.softlimit_in_bytes) - Desired memory
MemoryMinimumGaurantee - Minimum memory required (without this amount of
memory, VM should not be started)
SwapHardLimits (memory.memsw_limit_in_bytes) - Maximum swap
SwapSoftLimits (Currently not supported by kernel) - Desired swap space
Tunables memory.limit_in_bytes, memory.softlimit_in_bytes and
memory.memsw_limit_in_bytes are provided by the memory controller in the
Linux kernel.
I am not an expert here, so just listing what new elements need to be added
to the XML schema:
<define name="resource">
<element memory>
<element memoryHardLimit/>
<element memorySoftLimit/>
<element memoryMinGaurantee/>
<element swapHardLimit/>
<element swapSoftLimit/>
</element>
</define>
Pros:
* Support all the tunables exported by the kernel
* More tunables can be added as and when required
Cons:
* Code changes would touch various levels
* Might need to redefine(changing the scope) of existing memory
API. Currently, domainSetMemory is used to set limit_in_bytes in LXC and
memory ballooning in QEmu. While the domainSetMaxMemory is not defined in
QEmu and in case of LXC it is setting the internal object's maxmem
variable.
Future:
* Later on, CPU/IO/Network controllers related tunables can be
added/enhanced along with the APIs/XML elements:
CPUHardLimit
CPUSoftLimit
CPUShare
CPUPercentage
IO_BW_Softlimit
IO_BW_Hardlimit
IO_BW_percentage
* libvirt-cim support for resource management
Design 2: Reuse the current memory APIs in libvirt
==================================================
Use memory.limit_in_bytes to tweak memory hard limits
Init - Set the memory.limit_in_bytes to maximum mem.
Claiming memory from guest:
a) Reduce balloon size
b) If the guest does not co-operate(How do we know?), reduce
memory.limit_in_bytes.
Allocating memory more than max memory: How to solve this? As we have
already set the max balloon size. We can only play within this!
Pros:
* Few changes
* Is not intrusive
Cons:
* SetMemory and SetMaxMemory usage is confusing.
* SetMemory is too generic a name, it does not cover all the tunables.
* Does not support memory softlimit
* Does not have support to reserve the memory swap region
* This solution is not extensible
IMO, "Design 1" is more generic and extensible for various memory
tuneables.
Nikunj
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list