[Cc Frederic]
On Thu, Mar 07, 2024 at 05:09:51PM +0100, Stefan Wiehler wrote:
> With CONFIG_PROVE_RCU_LIST=y and by executing
>
> $ echo 0 > /sys/devices/system/cpu/cpu1/online
>
> one can trigger the following Lockdep-RCU splat on ARM:
>
> =============================
> WARNING: suspicious RCU usage
> 6.8.0-rc7-00001-g0db1d0ed8958 #10 Not tainted
> -----------------------------
> kernel/locking/lockdep.c:3762 RCU-list traversed in non-reader section!!
>
> other info that might help us debug this:
>
> RCU used illegally from offline CPU!
> rcu_scheduler_active = 2, debug_locks = 1
> no locks held by swapper/1/0.
>
> stack backtrace:
> CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.8.0-rc7-00001-g0db1d0ed8958 #10
> Hardware name: Allwinner sun8i Family
> unwind_backtrace from show_stack+0x10/0x14
> show_stack from dump_stack_lvl+0x60/0x90
> dump_stack_lvl from lockdep_rcu_suspicious+0x150/0x1a0
> lockdep_rcu_suspicious from __lock_acquire+0x11fc/0x29f8
> __lock_acquire from lock_acquire+0x10c/0x348
> lock_acquire from _raw_spin_lock_irqsave+0x50/0x6c
> _raw_spin_lock_irqsave from check_and_switch_context+0x7c/0x4a8
> check_and_switch_context from arch_cpu_idle_dead+0x10/0x7c
> arch_cpu_idle_dead from do_idle+0xbc/0x138
> do_idle from cpu_startup_entry+0x28/0x2c
> cpu_startup_entry from secondary_start_kernel+0x11c/0x124
> secondary_start_kernel from 0x401018a0
>
> The CPU is already reported as offline from RCU perspective in
> cpuhp_report_idle_dead() before arch_cpu_idle_dead() is invoked. Above
> RCU-Lockdep splat is then triggered by check_and_switch_context() acquiring the
> ASID spinlock.
>
> Avoid the false-positive Lockdep-RCU splat by briefly reporting the CPU as
I'm not sure this is a false-positive: if you traverse an RCU-list
without RCU watching the CPU, then a grace period may pass, the next
list can be garbage and you can go anywhere from the list. Or am I
missing something that a CPU hotplug can block a grace period?
But codewise, it looks good to me. Although I hope we can avoid calling
rcutree_report_cpu_{starting,dead}() here and use something simple,
however after a few attempts, I don't find a better way.
Regards,
Boqun
> online again while the spinlock is held.
>
> Signed-off-by: Stefan Wiehler <stefan.wiehler@xxxxxxxxx>
> ---
> arch/arm/kernel/smp.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index 3431c0553f45..6875e2c5dd50 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
> @@ -319,7 +319,14 @@ void __noreturn arch_cpu_idle_dead(void)
> {
> unsigned int cpu = smp_processor_id();
>
> + /*
> + * Briefly report CPU as online again to avoid false positive
> + * Lockdep-RCU splat when check_and_switch_context() acquires ASID
> + * spinlock.
> + */
> + rcutree_report_cpu_starting(cpu);
> idle_task_exit();
> + rcutree_report_cpu_dead();
>
> local_irq_disable();
>
> --
> 2.42.0
>
