On Fri, Oct 13, 2023 at 12:03:18PM -0400, Waiman Long <longman@xxxxxxxxxx> wrote: > > [chain] > > root > > | \ > > mid1a mid1b > > cpuset.cpus=0-1 cpuset.cpus=2-15 > > cpuset.cpus.partition=root > > | > > mid2 > > cpuset.cpus=0-1 > > cpuset.cpus.partition=root > > | > > cont > > cpuset.cpus=0-1 > > cpuset.cpus.partition=root > In this case, the effective CPUs of both mid1a and mid2 will be empty. IOW, > you can't have any task in these 2 cpusets. I see, that is relevant to a threaded subtree only where the admin / app can know how to distribute CPUs and place threads to internal nodes. > For the remote case, you can have intermediate tasks in both mid1a and mid2 > as long as cpuset.cpus contains more CPUs than cpuset.cpus.exclusive. It's obvious that cpuset.cpus.exclusive should be exclusive among siblings. Should it also be so along the vertical path? root | mid1a cpuset.cpus=0-2 cpuset.cpus.exclusive=0 | mid2 cpuset.cpus=0-2 cpuset.cpus.exclusive=1 | cont cpuset.cpus=0-2 cpuset.cpus.exclusive=2 cpuset.cpus.partition=root IIUC, this should be a valid config regardless of cpuset.cpus.partition setting on mid1a and mid2. Whereas root | mid1a cpuset.cpus=0-2 cpuset.cpus.exclusive=0 | mid2 cpuset.cpus=0-2 cpuset.cpus.exclusive=1-2 cpuset.cpus.partition=root | cont cpuset.cpus=1-2 cpuset.cpus.exclusive=1-2 cpuset.cpus.partition=root Here, I'm hesitating, will mid2 have any exclusively owned cpus? (I have flashes of understading cpus.exclusive as being a more expressive mechanism than partitions. OTOH, it seems non-intuitive when both are combined, thus I'm asking to internalize it better. Should partitions be deprecated for simplicty? They're still good to provide the notification mechanism of invalidation. cpuset.cpus.exclusive.effective don't have that.) > They will be ready eventually. This requirement of remote partition actually > came from our OpenShift team as the use of just local partition did not meet > their need. They don't need access to exclusive CPUs in the parent cgroup > layer for their management daemons. They do need to activate isolated > partition in selected child cgroups to support our Telco customers to run > workloads like DPDK. > > So they will add the support to upstream Kubernetes. Is it worth implementing anything touching (ancestral) cpuset.cpus.partition then? Thanks, Michal
