Hi libvirt experts,
---------- what does this patch support ----------
This patch seeks the support of system default memory policy
when numatune is configured. Before this patch, numatune only
has three memory modes: static, interleave and prefered.
These memory policies are ultimately set by mbind() system call.
Memory policy could be 'hard coded' into the kernel, but none of
above policies fit our requirment under this case. mbind() support
default memory policy, but it requires a NULL nodemask. So obviously
setting allowed memory nodes is cgroups' mission. That means if
'default' mode is specified in numatune, numatune config will be
completely cgroups setting, which restricting the memory
nodes allowed for each vcpu thread in different cells.
<numatune>
<memory mode="default" nodeset="1-4,^3" migratable="yes" />
<memnode cellid="0" mode="default" nodeset="1"/>
<memnode cellid="2" mode="default" nodeset="2"/>
</numatune>
Above is current design, we introduce a new "migratable" attribute and
a new option "default" for "mode" attribute in numatune.
---------- summary of discussion ----------
But at the beginning in RFC[1], Martin Kletzander and I got a consensus,
just introducing a new "migratable" attribute to indicates that we rely
on operating system to operate memory policy.
<numatune>
<memory mode="strict" nodeset="1-4,^3" migratable="yes" />
<memnode cellid="0" mode="strict" nodeset="1"/>
<memnode cellid="2" mode="preferred" nodeset="2"/>
</numatune>
the mode will be ignored, and libvirt will not set those memory policy
related configs in qemu command line.
Then initial patch got Peter Krempa's comments[2], if I understood it
correctly, that means config should not be ignored directly (I just stated
in document it will be ignored), it should be rejected in code and output
log to lead user correct the config.
But when I updated the patch, I found I had to add a new option 'default'
for mode attribute since its default value is "strict", if not setting
mode it will still be formatted to "strict" at the end. So I sent a RFC[3]
again, but unfortunately it sunk into the mail sea.
(sorry for the RFC/PATCH title changing)
[1] https://www.redhat.com/archives/libvir-list/2020-August/msg00960.html
[2] https://www.redhat.com/archives/libvir-list/2020-October/msg00396.html
[3] https://www.redhat.com/archives/libvir-list/2020-October/msg01033.html
mbind(): https://man7.org/linux/man-pages/man2/mbind.2.html
Though we have "migratable" in current implementation, actually we can
support this feauture without it since we have "default" mode, so is
"migratable" still neccessary?
Thanks for your patience to read this. Appreciate if you could give comments.
Regards,
Luyao
Luyao Zhong (1):
support system default memory policy with numatune
docs/formatdomain.rst | 12 ++++-
docs/schemas/domaincommon.rng | 7 +++
include/libvirt/libvirt-domain.h | 1 +
src/conf/numa_conf.c | 50 ++++++++++++++++++-
src/conf/numa_conf.h | 2 +
src/libvirt_private.syms | 1 +
src/qemu/qemu_command.c | 7 ++-
src/qemu/qemu_process.c | 25 ++++++++++
src/util/virnuma.c | 3 ++
.../numatune-memory-migratable.args | 34 +++++++++++++
...atune-memory-migratable.x86_64-latest.args | 35 +++++++++++++
.../numatune-memory-migratable.xml | 33 ++++++++++++
tests/qemuxml2argvtest.c | 2 +
...matune-memory-migratable.x86_64-latest.xml | 40 +++++++++++++++
.../numatune-memory-migratable.xml | 39 +++++++++++++++
tests/qemuxml2xmltest.c | 1 +
16 files changed, 288 insertions(+), 4 deletions(-)
create mode 100644 tests/qemuxml2argvdata/numatune-memory-migratable.args
create mode 100644 tests/qemuxml2argvdata/numatune-memory-migratable.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/numatune-memory-migratable.xml
create mode 100644 tests/qemuxml2xmloutdata/numatune-memory-migratable.x86_64-latest.xml
create mode 100644 tests/qemuxml2xmloutdata/numatune-memory-migratable.xml
--
2.25.4
