Update Documentation/admin-guide/cgroup-v2.rst on the newly introduced
"isolated" cpuset partition type as well as the ability to create
non-top cpuset partition with no cpu allocated to it.
Signed-off-by: Waiman Long <longman@xxxxxxxxxx>
---
Documentation/admin-guide/cgroup-v2.rst | 104 +++++++++++++++---------
1 file changed, 64 insertions(+), 40 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 5c7377b5bd3e..a8faaba1950e 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2080,8 +2080,9 @@ Cpuset Interface Files
It accepts only the following input values when written to.
======== ================================
- "root" a partition root
- "member" a non-root member of a partition
+ "member" Non-root member of a partition
+ "root" Partition root
+ "isolated" Partition root without load balancing
======== ================================
When set to be a partition root, the current cgroup is the
@@ -2090,9 +2091,14 @@ Cpuset Interface Files
partition roots themselves and their descendants. The root
cgroup is always a partition root.
- There are constraints on where a partition root can be set.
- It can only be set in a cgroup if all the following conditions
- are true.
+ When set to "isolated", the CPUs in that partition root will
+ be in an isolated state without any load balancing from the
+ scheduler. Tasks in such a partition must be explicitly bound
+ to each individual CPU.
+
+ There are constraints on where a partition root can be set
+ ("root" or "isolated"). It can only be set in a cgroup if all
+ the following conditions are true.
1) The "cpuset.cpus" is not empty and the list of CPUs are
exclusive, i.e. they are not shared by any of its siblings.
@@ -2103,51 +2109,69 @@ Cpuset Interface Files
eliminating corner cases that have to be handled if such a
condition is allowed.
- Setting it to partition root will take the CPUs away from the
- effective CPUs of the parent cgroup. Once it is set, this
+ Setting it to a partition root will take the CPUs away from
+ the effective CPUs of the parent cgroup. Once it is set, this
file cannot be reverted back to "member" if there are any child
cgroups with cpuset enabled.
- A parent partition cannot distribute all its CPUs to its
- child partitions. There must be at least one cpu left in the
- parent partition.
-
- Once becoming a partition root, changes to "cpuset.cpus" is
- generally allowed as long as the first condition above is true,
- the change will not take away all the CPUs from the parent
- partition and the new "cpuset.cpus" value is a superset of its
- children's "cpuset.cpus" values.
-
- Sometimes, external factors like changes to ancestors'
- "cpuset.cpus" or cpu hotplug can cause the state of the partition
- root to change. On read, the "cpuset.sched.partition" file
- can show the following values.
+ A parent partition may distribute all its CPUs to its child
+ partitions as long as it is not the root cgroup and there is no
+ task directly associated with that parent partition. Otherwise,
+ there must be at least one cpu left in the parent partition.
+ A new task cannot be moved to a partition root with no effective
+ cpu.
+
+ Once becoming a partition root, changes to "cpuset.cpus"
+ is generally allowed as long as the first condition above
+ (cpu exclusivity rule) is true.
+
+ Sometimes, changes to "cpuset.cpus" or cpu hotplug may cause
+ the state of the partition root to become invalid when the
+ other constraints of partition root are violated. Therefore,
+ it is recommended that users should always set "cpuset.cpus"
+ to the proper value first before enabling partition. In case
+ "cpuset.cpus" has to be modified after partition is enabled,
+ users should check the state of "cpuset.cpus.partition" after
+ making change to it to make sure that the partition is still
+ valid.
+
+ On read, the "cpuset.cpus.partition" file can show the following
+ values.
============== ==============================
"member" Non-root member of a partition
"root" Partition root
+ "isolated" Partition root without load balancing
"root invalid" Invalid partition root
============== ==============================
- It is a partition root if the first 2 partition root conditions
- above are true and at least one CPU from "cpuset.cpus" is
- granted by the parent cgroup.
-
- A partition root can become invalid if none of CPUs requested
- in "cpuset.cpus" can be granted by the parent cgroup or the
- parent cgroup is no longer a partition root itself. In this
- case, it is not a real partition even though the restriction
- of the first partition root condition above will still apply.
- The cpu affinity of all the tasks in the cgroup will then be
- associated with CPUs in the nearest ancestor partition.
-
- An invalid partition root can be transitioned back to a
- real partition root if at least one of the requested CPUs
- can now be granted by its parent. In this case, the cpu
- affinity of all the tasks in the formerly invalid partition
- will be associated to the CPUs of the newly formed partition.
- Changing the partition state of an invalid partition root to
- "member" is always allowed even if child cpusets are present.
+ A partition root becomes invalid if all the CPUs requested in
+ "cpuset.cpus" become unavailable. This can happen if all the
+ CPUs have been offlined, or the state of an ancestor partition
+ root become invalid. In this case, it is not a real partition
+ even though the restriction of the cpu exclusivity rule will
+ still apply. The cpu affinity of all the tasks in the cgroup
+ will then be associated with CPUs in the nearest ancestor
+ partition.
+
+ In the special case of a parent partition competing with a child
+ partition for the only CPU left, the parent partition wins and
+ the child partition becomes invalid.
+
+ An invalid partition root can be transitioned back to a real
+ partition root if at least one of the requested CPUs become
+ available again. In this case, the cpu affinity of all the tasks
+ in the formerly invalid partition will be associated to the CPUs
+ of the newly formed partition. Changing the partition state of
+ an invalid partition root to "member" is always allowed even if
+ child cpusets are present. However changing a partition root back
+ to member will not be allowed if child partitions are present.
+
+ Poll and inotify events are triggered whenever the state
+ of "cpuset.cpus.partition" changes. That includes changes
+ caused by write to "cpuset.cpus.partition" and cpu hotplug.
+ This will allow a user space agent to monitor changes caused
+ by hotplug events.
Device controller
--
2.18.1
