On Thu, Mar 07, 2024 at 05:09:51PM +0100, Stefan Wiehler wrote:
> With CONFIG_PROVE_RCU_LIST=y and by executing
>
> $ echo 0 > /sys/devices/system/cpu/cpu1/online
>
> one can trigger the following Lockdep-RCU splat on ARM:
>
> =============================
> WARNING: suspicious RCU usage
> 6.8.0-rc7-00001-g0db1d0ed8958 #10 Not tainted
> -----------------------------
> kernel/locking/lockdep.c:3762 RCU-list traversed in non-reader section!!
>
> other info that might help us debug this:
>
> RCU used illegally from offline CPU!
> rcu_scheduler_active = 2, debug_locks = 1
> no locks held by swapper/1/0.
>
> stack backtrace:
> CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.8.0-rc7-00001-g0db1d0ed8958 #10
> Hardware name: Allwinner sun8i Family
> unwind_backtrace from show_stack+0x10/0x14
> show_stack from dump_stack_lvl+0x60/0x90
> dump_stack_lvl from lockdep_rcu_suspicious+0x150/0x1a0
> lockdep_rcu_suspicious from __lock_acquire+0x11fc/0x29f8
> __lock_acquire from lock_acquire+0x10c/0x348
> lock_acquire from _raw_spin_lock_irqsave+0x50/0x6c
> _raw_spin_lock_irqsave from check_and_switch_context+0x7c/0x4a8
> check_and_switch_context from arch_cpu_idle_dead+0x10/0x7c
> arch_cpu_idle_dead from do_idle+0xbc/0x138
> do_idle from cpu_startup_entry+0x28/0x2c
> cpu_startup_entry from secondary_start_kernel+0x11c/0x124
> secondary_start_kernel from 0x401018a0
>
> The CPU is already reported as offline from RCU perspective in
> cpuhp_report_idle_dead() before arch_cpu_idle_dead() is invoked. Above
> RCU-Lockdep splat is then triggered by check_and_switch_context() acquiring the
> ASID spinlock.
>
> Avoid the false-positive Lockdep-RCU splat by briefly reporting the CPU as
> online again while the spinlock is held.
>
> Signed-off-by: Stefan Wiehler <stefan.wiehler@xxxxxxxxx>
>From an RCU perspective, this looks plausible. One question
below.
Thanx, Paul
> ---
> arch/arm/kernel/smp.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index 3431c0553f45..6875e2c5dd50 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
> @@ -319,7 +319,14 @@ void __noreturn arch_cpu_idle_dead(void)
> {
> unsigned int cpu = smp_processor_id();
>
> + /*
> + * Briefly report CPU as online again to avoid false positive
> + * Lockdep-RCU splat when check_and_switch_context() acquires ASID
> + * spinlock.
> + */
> + rcutree_report_cpu_starting(cpu);
> idle_task_exit();
> + rcutree_report_cpu_dead();
>
> local_irq_disable();
Both rcutree_report_cpu_starting() and rcutree_report_cpu_dead() complain
bitterly via lockdep if interrupts are enabled. And the call sites have
interrupts disabled. So I don't understand what this local_irq_disable()
is needed for.
