On Fri, Jan 19, 2024 at 11:08:23PM +0100, Frederic Weisbecker wrote: > Le Fri, Jan 19, 2024 at 04:38:23PM +0100, Leon Woestenberg a écrit : > > Hello all, > > > > On 6.1.71-rt21 (PREEMPT_RT against vanilla) I am trying to find the > > root cause of an "rcuc" thread scheduled in on an isolated core (CPU > > #4). > > The isolated core should run a single non-kernel user-space-busy-loop > > thread (SCHED_OTHER, not FIFO or RR). > > > > tsc=reliable isolcpus=domain,managed_irq,4-7 nohz_full=4-7 > > rcu_nocbs=all rcu_nocb_poll > > /proc/irq/*/smp_affinity and cpumasks in > > /sys/devices/virtual/workqueue/ all show 000f or less (CPUs 0-3) > > > > Using osnoise I see rcuc/4 popping up as a cause of noise, but my > > kernel command line should prevent that, right? Well, your choice is rcuc or ksoftirqd. Either way, there is some per-CPU work that must be done. But to Frederic's later point, not very much work. In particular, RCU callbacks should be invoked on an rcuoc kthread. Unless CPU 4 spends absolutely no time in the kernel, and never ever has more than one runnable task at any given time. And doesn't have any other excuse to take a scheduling-clock interrupt. Otherwise, CPU 4's rcuc kthread will burn at least a little bit of time. The more time CPU 4 spends in the kernel and the more time CPU 4 spends with more than one runnable task, the more time CPU 4's rcuc kthread will burn. > > I see this in the kernel boot log, and I wonder if this is related to my issue: > > > > [ 0.063518] rcu: RCU_SOFTIRQ processing moved to rcuc kthreads. This simply means that the processing that would have been done in softirq (for example, by ksoftirqd) will now be done by rcuc. > > tree.c:102:static bool use_softirq = !IS_ENABLED(CONFIG_PREEMPT_RT); > > > > Could this cause the rcuc threads to become active, even with the > > "rcu_nocbs=all rcu_nocb_poll" setting? You can override this with rcutree.use_softirq=1 if you wish. But that will simply move the processing from the rcuc kthread to softirq context. > That's unexpected. The RCU load should be moved from rcuc to rcuo* kthreads. > > Do you have CONFIG_RCU_STRICT_GRACE_PERIOD=y perhaps? > > Also do you really need rcu_nocb_poll? I would expect this option to be useful > for debugging only, though I'm not aware of every usecases. I have the same questions. ;-) Thanx, Paul > Thanks. > > > > > > > Thanks, Leon. > > > > > > [ 0.063516] rcu: Preemptible hierarchical RCU implementation. > > [ 0.063517] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=16. > > [ 0.063518] rcu: RCU priority boosting: priority 1 delay 500 ms. > > [ 0.063518] rcu: RCU_SOFTIRQ processing moved to rcuc kthreads. > > [ 0.063519] No expedited grace period (rcu_normal_after_boot). > > [ 0.063519] Trampoline variant of Tasks RCU enabled. > > [ 0.063519] Rude variant of Tasks RCU enabled. > > [ 0.063520] Tracing variant of Tasks RCU enabled. > > [ 0.063520] rcu: RCU calculated value of scheduler-enlistment delay > > is 25 jiffies. > > [ 0.063521] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=16 > > [ 0.066896] NO_HZ: Full dynticks CPUs: 4-7. > > [ 0.066897] rcu: Offload RCU callbacks from CPUs: 0-15. > > [ 0.066898] rcu: Poll for callbacks from no-CBs CPUs. > > [ 0.066901] rcu: srcu_init: Setting srcu_struct sizes based on contention. > > [ 1.145094] rcu: Hierarchical SRCU implementation. > > [ 1.145094] rcu: Max phase no-delay instances is 1000. > > > > > > -- > > Leon Woestenberg > > leon@xxxxxxxxxxxxxx > > T: +31 40 711 42 76 > > M: +31 6 472 30 372 > > > > Sidebranch Embedded Systems > > Eindhoven, The Netherlands > > http://www.sidebranch.com > >
