Update Documentation/admin-guide/cgroup-v2.rst on the newly introduced
"isolated" cpuset partition type as well as other changes made in other
cpuset patches.
Signed-off-by: Waiman Long <longman@xxxxxxxxxx>
---
Documentation/admin-guide/cgroup-v2.rst | 168 ++++++++++++++----------
1 file changed, 102 insertions(+), 66 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 2aeb7ae8b393..9612319b353f 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2099,74 +2099,110 @@ Cpuset Interface Files
It accepts only the following input values when written to.
======== ================================
- "root" a partition root
- "member" a non-root member of a partition
+ "member" Non-root member of a partition
+ "root" Partition root
+ "isolated" Partition root without load balancing
======== ================================
- When set to be a partition root, the current cgroup is the
- root of a new partition or scheduling domain that comprises
- itself and all its descendants except those that are separate
- partition roots themselves and their descendants. The root
- cgroup is always a partition root.
-
- There are constraints on where a partition root can be set.
- It can only be set in a cgroup if all the following conditions
- are true.
-
- 1) The "cpuset.cpus" is not empty and the list of CPUs are
- exclusive, i.e. they are not shared by any of its siblings.
- 2) The parent cgroup is a partition root.
- 3) The "cpuset.cpus" is also a proper subset of the parent's
- "cpuset.cpus.effective".
- 4) There is no child cgroups with cpuset enabled. This is for
- eliminating corner cases that have to be handled if such a
- condition is allowed.
-
- Setting it to partition root will take the CPUs away from the
- effective CPUs of the parent cgroup. Once it is set, this
- file cannot be reverted back to "member" if there are any child
- cgroups with cpuset enabled.
-
- A parent partition cannot distribute all its CPUs to its
- child partitions. There must be at least one cpu left in the
- parent partition.
-
- Once becoming a partition root, changes to "cpuset.cpus" is
- generally allowed as long as the first condition above is true,
- the change will not take away all the CPUs from the parent
- partition and the new "cpuset.cpus" value is a superset of its
- children's "cpuset.cpus" values.
-
- Sometimes, external factors like changes to ancestors'
- "cpuset.cpus" or cpu hotplug can cause the state of the partition
- root to change. On read, the "cpuset.sched.partition" file
- can show the following values.
-
- ============== ==============================
- "member" Non-root member of a partition
- "root" Partition root
- "root invalid" Invalid partition root
- ============== ==============================
-
- It is a partition root if the first 2 partition root conditions
- above are true and at least one CPU from "cpuset.cpus" is
- granted by the parent cgroup.
-
- A partition root can become invalid if none of CPUs requested
- in "cpuset.cpus" can be granted by the parent cgroup or the
- parent cgroup is no longer a partition root itself. In this
- case, it is not a real partition even though the restriction
- of the first partition root condition above will still apply.
- The cpu affinity of all the tasks in the cgroup will then be
- associated with CPUs in the nearest ancestor partition.
-
- An invalid partition root can be transitioned back to a
- real partition root if at least one of the requested CPUs
- can now be granted by its parent. In this case, the cpu
- affinity of all the tasks in the formerly invalid partition
- will be associated to the CPUs of the newly formed partition.
- Changing the partition state of an invalid partition root to
- "member" is always allowed even if child cpusets are present.
+ The root cgroup is always a partition root and its state
+ cannot be changed. All other non-root cgroups start out as
+ "member".
+
+ When set to "root", the current cgroup is the root of a new
+ partition or scheduling domain that comprises itself and
+ all its descendants except those that are separate partition
+ roots themselves and their descendants.
+
+ The value shown in "cpuset.cpus.effective" of a partition root is
+ the CPUs that the parent partition root can dedicate to the new
+ partition root. They are subtracted from "cpuset.cpus.effective"
+ of the parent and may be different from "cpuset.cpus"
+
+ When set to "isolated", the CPUs in that partition root will
+ be in an isolated state without any load balancing from the
+ scheduler. Tasks placed in such a partition with multiple
+ CPUs should be carefully distributed and bound to each of the
+ individual CPUs for optimal performance.
+
+ A partition root ("root" or "isolated") can be in one of the
+ two possible states - valid or invalid. An invalid partition
+ root is in a degraded state where some state information are
+ retained, but behaves more like a "member".
+
+ On read, the "cpuset.cpus.partition" file can show the following
+ values.
+
+ ====================== ==============================
+ "member" Non-root member of a partition
+ "root" Partition root
+ "isolated" Partition root without load balancing
+ "root invalid (<reason>)" Invalid partition root
+ "isolated invalid (<reason>)" Invalid isolated partition root
+ ====================== ==============================
+
+ In the case of an invalid partition root, a descriptive string on
+ why the partition is invalid is included within parentheses.
+
+ Almost all possible state transitions among "member", valid
+ and invalid partition roots are allowed except from "member"
+ to invalid partition root.
+
+ Before the "member" to partition root transition can happen,
+ the following conditions must be met or the transition will
+ not be allowed.
+
+ 1) The "cpuset.cpus" is non-empty and exclusive, i.e. they are
+ not shared by any of its siblings.
+ 2) The parent cgroup is a valid partition root.
+ 3) The "cpuset.cpus" must contain at least one of the CPUs from
+ parent's "cpuset.cpus", i.e. they overlap.
+ 4) There is no child cgroups with cpuset enabled. This avoids
+ cpu migrations of multiple cgroups simultaneously which can
+ be problematic.
+
+ Once becoming a partition root, the only rule restricting
+ changes made to "cpuset.cpus" is the exclusivity rule where
+ none of the siblings of a partition root can share CPUs with
+ it.
+
+ External events like hotplug or inappropriate changes to
+ "cpuset.cpus" can cause a valid partition root to become invalid.
+ Besides the exclusivity rule listed above, the other conditions
+ required to maintain the validity of a partition root are
+ as follows:
+
+ 1) The parent cgroup is a valid partition root.
+ 2) If "cpuset.cpus.effective" is empty, the partition must have
+ no task associated with it. Otherwise, the partition becomes
+ invalid and "cpuset.cpus.effective" will fall back to that
+ of the nearest non-empty ancestor.
+
+ A corollary of a valid partition root is that
+ "cpuset.cpus.effective" is always a subset of "cpuset.cpus".
+ Note that a task cannot be moved to a cgroup with empty
+ "cpuset.cpus.effective".
+
+ A valid non-root parent partition may distribute out all its CPUs
+ to its child partitions when there is no task associated with it.
+
+ An invalid partition root will be reverted back to a valid
+ one if none of the validity constraints of a valid partition
+ root are violated due to hotplug events or proper changes to
+ "cpuset.cpus" files.
+
+ Changing a partition root (valid or invalid) to "member" is
+ always allowed. If there are child partition roots underneath
+ it, they will become invalid and unrecoverable. So care must
+ be taken to double check for this condition before disabling
+ a partition root.
+
+ Poll and inotify events are triggered whenever the state of
+ "cpuset.cpus.partition" changes. That includes changes caused
+ by write to "cpuset.cpus.partition", cpu hotplug or other
+ changes that modify the validity status of the partition.
+ This will allow user space agents to monitor unexpected changes
+ to "cpuset.cpus.partition" without the need to do continuous
+ polling.
Device controller
--
2.27.0
