On Wed, Sep 14, 2022 at 06:12:27PM +0800, Pingfan Liu wrote:
> On Sat, Sep 10, 2022 at 01:36:22PM -0700, Paul E. McKenney wrote:
> Actually, the scheduler will pick up one online cpu for the boost
> thread.
>
> On the last cpu, boost is subject to migration, and is pushed away by
> __balance_push_cpu_stop()
> {
> if (task_rq(p) == rq && task_on_rq_queued(p)) {
> cpu = select_fallback_rq(rq->cpu, p);
> rq = __migrate_task(rq, &rf, p, cpu);
> }
> }
>
> Now, turning to select_fallback_rq(), inside that function, if
> p->cpus_ptr has no suitable cpus, then case cpuset or possible will make it
> point to cpuset or cpu_possible_mask. So finally, there is a valid cpu
> is picked up. (I have not found the online cpu there. The trick should hide
> elsewhere. But that rendered the same result as in v5.16, where
> rcu_boost_kthread_setaffinity()->cpumask_setall(cm))
>
> But now, in 6.0, it changes into housekeeping_cpumask(HK_TYPE_RCU),
> which appeals for a more careful thinking.
> If there is a cpuset for housekeeping so that select_fallback_rq() can
> pick up one from that cpuset?
>
> To Frederic, could you show some light here and is it worth introducing
> such a cpuset so that select_fallback_rq() can pick up a cpu in
> housekeeping set in this case?
>
>
> Anyway, as the last resort, TICK_DEP_BIT_RCU has already appealed for
> the awareness of the concurrent rcutree_offline_cpu(), the affinity
> setting could be done there if select_fallback_rq() can not work.
Here is the way I understand it: suppose the outgoing CPU was the last online
one in this rcu node, in this case the migration performed in
sched_cpu_deactivate() will find only one CPU in p->cpus_mask, but that CPU is
now out of cpu_active_mask, so the task is affined by default to cpu_possible_mask.
So there is a short window between sched_cpu_deactivate() and
rcutree_offline_cpu() where the task may run on all cpu_possible_mask. Does it
matter? Maybe, I don't know...
After that we reach rcutree_offline_cpu() -> rcu_boost_kthread_setaffinity()
that fails to fill the cpumask since the node is now empty. As the resulting
cpumask is empty, it fills it as a last resort to the HK_TYPE_RCU housekeeping
set which has to have at least 1 online CPU (nohz_full constraint). So the task
is finally affine to that housekeeping set.
Like Paul said, I'd rather indeed remove the rcu_boost_kthread_setaffinity()
call from rcutree_dead_cpu() as this one looks useless.
Thanks.
>
> Thanks,
>
> Pingfan
