On 1/12/23 13:03, Daniel P. Berrangé wrote:
On Thu, Jan 05, 2023 at 03:53:10PM +0100, Pierre Morel wrote:
The modification of the CPU attributes are done through a monitor
commands.
It allows to move the core inside the topology tree to optimise
the cache usage in the case the host's hypervizor previously
moved the CPU.
The same command allows to modifiy the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
Signed-off-by: Pierre Morel <pmorel@xxxxxxxxxxxxx>
---
qapi/machine-target.json | 29 ++++++++
include/monitor/hmp.h | 1 +
hw/s390x/cpu-topology.c | 141 +++++++++++++++++++++++++++++++++++++++
hmp-commands.hx | 16 +++++
4 files changed, 187 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 2e267fa458..75b0aa254d 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -342,3 +342,32 @@
'TARGET_S390X',
'TARGET_MIPS',
'TARGET_LOONGARCH64' ] } }
+
+##
+# @change-topology:
+#
+# @core: the vCPU ID to be moved
+# @socket: the destination socket where to move the vCPU
+# @book: the destination book where to move the vCPU
+# @drawer: the destination drawer where to move the vCPU
This movement can be done while the guest OS is running ?
What happens to guest OS apps ? Every I know will read
topology once and assume it never changes at runtime.
Yes this can change while the guest is running.
The S390 Logical PARtition, where the Linux runs is already a first
level of virtualization and the lpar CPU are already virtual CPU which
can be moved from one real CPU to another, the guest is at a second
level of virtualization.
On the LPAR host an admin can check the topology.
A lpar CPU can be moved to another real CPU because of multiple reasons:
maintenance, failure, other decision from the first level hypervisor
that I do not know, may be scheduling balancing.
There is a mechanism for the OS in which is running in LPAR to set a
flag for the guest on a topology change.
The guest use a specific instruction to get this flag.
This instruction PTF(2) is interpreted by the firmware and does not
appear in this patch series but in Linux patch series.
So we have, real CPU <-> lpar CPU <-> vCPU
What's the use case for wanting to re-arrange topology in
this manner ? It feels like its going to be a recipe for
hard to diagnose problems, as much code in libvirt and apps
above will assuming the vCPU IDs are assigned sequentially
starting from node=0,book=0,drawer=0,socket=0,core=0,
incrementing core, then incrementing socket, then
incrementing drawer, etc.
The goal to rearrange the vCPU is to give the guest the knowledge of the
topology so it can takes benefit of it.
If a lpar CPU moved to another real CPU in another drawer we must move
the guest vCPU to another drawer so the guest OS can take the best
scheduling decisions.
Per default, if nothing is specified on the creation of a vCPU, the
creation is done exactly like you said, starting from (0,0,0,0) and
incrementing.
There are two possibility to set a vCPU at its place:
1) on creation by specifying the drawer,book,socket for a specific core-id
2) with this QAPI command to move the CPU while it is running.
Note that the core-id and the CPU address do not change when moving the
CPU so that there is no problem with scheduling, all we do is to provide
the topology up to the guest when it asks.
The period of checking by the Linux kernel if there is a change and if
there is a need to ask the topology is one minute.
The migration of CPU is not supposed to happen very often, (not every day).
+# @polarity: optional polarity, default is last polarity set by the guest
+# @dedicated: optional, if the vCPU is dedicated to a real CPU
+#
+# Modifies the topology by moving the CPU inside the topology
+# tree or by changing a modifier attribute of a CPU.
+#
+# Returns: Nothing on success, the reason on failure.
+#
+# Since: <next qemu stable release, eg. 1.0>
+##
+{ 'command': 'change-topology',
'set-cpu-topology'
OK, yes looks better.
+ 'data': {
+ 'core': 'int',
+ 'socket': 'int',
+ 'book': 'int',
+ 'drawer': 'int',
+ '*polarity': 'int',
+ '*dedicated': 'bool'
+ },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
With regards,
Daniel
Thanks,
Regards,
Pierre
--
Pierre Morel
IBM Lab Boeblingen
