On 5/2/23 18:27, Michal Koutný wrote:
On Tue, May 02, 2023 at 05:26:17PM -0400, Waiman Long <longman@xxxxxxxxxx> wrote:
In the new scheme, the available cpus are still directly passed down to a
descendant cgroup. However, isolated CPUs (or more generally CPUs dedicated
to a partition) have to be exclusive. So what the cpuset.cpus.reserve does
is to identify those exclusive CPUs that can be excluded from the
effective_cpus of the parent cgroups before they are claimed by a child
partition. Currently this is done automatically when a child partition is
created off a parent partition root. The new scheme will break it into 2
separate steps without the requirement that the parent of a partition has to
be a partition root itself.
new scheme
1st step:
echo C >p/cpuset.cpus.reserve
# p/cpuset.cpus.effective == A-C (1)
2nd step (claim):
echo C' >p/c/cpuset.cpus # C'⊆C
echo root >p/c/cpuset.cpus.partition
It is something like that. However, the current scheme of automatic
reservation is also supported, i.e. cpuset.cpus.reserve will be set
automatically when the child cgroup becomes a valid partition as long as
the cpuset.cpus.reserve file is not written to. This is for backward
compatibility.
Once it is written to, automatic mode will end and users have to
manually set it afterward.
current scheme
1st step (configure):
echo C >p/c/cpuset.cpus
2nd step (reserve & claim):
echo root >p/c/cpuset.cpus.partition
# p/cpuset.cpus.effective == A-C (2)
As long as p/c is unpopulated, (1) and (2) are equal situations.
Why is the (different) two step procedure needed?
Also the relaxation of requirement of a parent being a partition
confuses me -- if the parent is not a partition, i.e. it has no
exclusive ownership of CPUs but it can still "give" it to children -- is
child partition meant to be exclusive? (IOW can parent siblings reserve
some same CPUs?)
A valid partition root has exclusive ownership of its CPUs. That is a
rule that won't be changed. As a result, an incoming partition root
cannot claim CPUs that have been allocated to another partition. To
simplify thing, transition to a valid partition root is not possible if
any of the CPUs in its cpuset.cpus are not in the cpuset.cpus.reserve of
its ancestor or have been allocated to another partition. The partition
root simply becomes invalid.
The parent can virtually give the reserved CPUs from the root down the
hierarchy and a child can claim them once it becomes a partition root.
In manual mode, we need to check all the way up the hierarchy to the
root to figure out what CPUs in cpuset.cpus.reserve are valid. It has
higher overhead, but enabling partition is not a fast operation anyway.
Cheers,
Longman
