Re: [PATCH-cgroup 1/4] workqueue: Add workqueue_unbound_exclude_cpumask() to exclude CPUs from wq_unbound_cpumask

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 10/18/23 09:41, Waiman Long wrote:
On 10/18/23 05:24, Tejun Heo wrote:
Hello,

On Fri, Oct 13, 2023 at 02:11:19PM -0400, Waiman Long wrote:
When the "isolcpus" boot command line option is used to add a set
of isolated CPUs, those CPUs will be excluded automatically from
wq_unbound_cpumask to avoid running work functions from unbound
workqueues.

Recently cpuset has been extended to allow the creation of partitions
of isolated CPUs dynamically. To make it closer to the "isolcpus"
in functionality, the CPUs in those isolated cpuset partitions should be
excluded from wq_unbound_cpumask as well. This can be done currently by
explicitly writing to the workqueue's cpumask sysfs file after creating
the isolated partitions. However, this process can be error prone.
Ideally, the cpuset code should be allowed to request the workqueue code
to exclude those isolated CPUs from wq_unbound_cpumask so that this
operation can be done automatically and the isolated CPUs will be returned
back to wq_unbound_cpumask after the destructions of the isolated
cpuset partitions.

This patch adds a new workqueue_unbound_exclude_cpumask() to enable
that. This new function will exclude the specified isolated CPUs
from wq_unbound_cpumask. To be able to restore those isolated CPUs
back after the destruction of isolated cpuset partitions, a new
wq_user_unbound_cpumask is added to store the user provided unbound
cpumask either from the boot command line options or from writing to
the cpumask sysfs file. This new cpumask provides the basis for CPU
exclusion.
The behaviors around wq_unbound_cpumask is getting pretty inconsistent:

1. Housekeeping excludes isolated CPUs on boot but allows user to override
    it to include isolated CPUs afterwards.

2. If an unbound wq's cpumask doesn't have any intersection with
    wq_unbound_cpumask we ignore the per-wq cpumask and falls back to
    wq_unbound_cpumask.

3. You're adding a masking layer on top with exclude which fails to set if
    the intersection is empty.

Can we do the followings for consistency?

1. User's requested_unbound_cpumask is stored separately (as in this patch).

2. The effect wq_unbound_cpumask is determined by requested_unbound_cpumask
    & housekeeping_cpumask & cpuset_allowed_cpumask. The operation order
    matters. When an & operation yields an cpumask, the cpumask from the
    previous step is the effective one.
Sure. I will do that.

I have a second thought after taking a further look at that. First of all, cpuset_allowed_mask isn't relevant here and the mask can certainly contain offline CPUs. So cpu_possible_mask is the proper fallback.

With the current patch, wq_user_unbound_cpumask is set up initially as  (HK_TYPE_WQ ∩ HK_TYPE_DOMAIN) house keeping mask and rewritten by any subsequent write to workqueue/cpumask sysfs file. So using wq_user_unbound_cpumask has the implied precedence of user-sysfs written mask, command line isolcpus or nohz_full option mask and cpu_possible_mask. I think just fall back to wq_user_unbound_cpumask if the operation fails should be enough.

Cheers,
Longman





[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]     [Monitors]

  Powered by Linux