On Wed, Sep 14, 2022 at 8:27 PM Frederic Weisbecker <frederic@xxxxxxxxxx> wrote:
>
> On Wed, Sep 14, 2022 at 06:12:27PM +0800, Pingfan Liu wrote:
> > On Sat, Sep 10, 2022 at 01:36:22PM -0700, Paul E. McKenney wrote:
> > Actually, the scheduler will pick up one online cpu for the boost
> > thread.
> >
> > On the last cpu, boost is subject to migration, and is pushed away by
> > __balance_push_cpu_stop()
> > {
> > if (task_rq(p) == rq && task_on_rq_queued(p)) {
> > cpu = select_fallback_rq(rq->cpu, p);
> > rq = __migrate_task(rq, &rf, p, cpu);
> > }
> > }
> >
> > Now, turning to select_fallback_rq(), inside that function, if
> > p->cpus_ptr has no suitable cpus, then case cpuset or possible will make it
> > point to cpuset or cpu_possible_mask. So finally, there is a valid cpu
> > is picked up. (I have not found the online cpu there. The trick should hide
> > elsewhere. But that rendered the same result as in v5.16, where
> > rcu_boost_kthread_setaffinity()->cpumask_setall(cm))
> >
> > But now, in 6.0, it changes into housekeeping_cpumask(HK_TYPE_RCU),
> > which appeals for a more careful thinking.
> > If there is a cpuset for housekeeping so that select_fallback_rq() can
> > pick up one from that cpuset?
> >
> > To Frederic, could you show some light here and is it worth introducing
> > such a cpuset so that select_fallback_rq() can pick up a cpu in
> > housekeeping set in this case?
> >
> >
> > Anyway, as the last resort, TICK_DEP_BIT_RCU has already appealed for
> > the awareness of the concurrent rcutree_offline_cpu(), the affinity
> > setting could be done there if select_fallback_rq() can not work.
>
> Here is the way I understand it: suppose the outgoing CPU was the last online
> one in this rcu node, in this case the migration performed in
> sched_cpu_deactivate() will find only one CPU in p->cpus_mask, but that CPU is
> now out of cpu_active_mask, so the task is affined by default to cpu_possible_mask.
>
> So there is a short window between sched_cpu_deactivate() and
> rcutree_offline_cpu() where the task may run on all cpu_possible_mask. Does it
> matter? Maybe, I don't know...
>
OK, then it is pointless to introduce a housekeeping cpuset.
> After that we reach rcutree_offline_cpu() -> rcu_boost_kthread_setaffinity()
> that fails to fill the cpumask since the node is now empty. As the resulting
> cpumask is empty, it fills it as a last resort to the HK_TYPE_RCU housekeeping
> set which has to have at least 1 online CPU (nohz_full constraint). So the task
> is finally affine to that housekeeping set.
>
Yes, exactly.
> Like Paul said, I'd rather indeed remove the rcu_boost_kthread_setaffinity()
> call from rcutree_dead_cpu() as this one looks useless.
>
OK, I will take that way.
Thanks to both of you for the generous help. I will bring up V2 soon.
Thanks,
Pingfan
