> > Hello, Paul > > when running the rcutorture test with rcutorture.object_debug=1 in > PREEMPT_RT kernel. > > BUG: sleeping function called from invalid context at > kernel/locking/spinlock_rt.c:48 > [ 3.116503] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: > 1, name: swapper/0 > [ 3.116503] preempt_count: 1, expected: 0 > [ 3.116503] RCU nest depth: 1, expected: 1 > [ 3.116503] 3 locks held by swapper/0/1: > [ 3.116503] #0: ffffffff8f862d80 (fullstop_mutex){+.+.}-{4:4}, at: > torture_init_begin+0x24/0xa0 > [ 3.116503] #1: ffffffff8f7b1400 (rcu_read_lock){....}-{1:3}, at: > rcu_torture_init+0x20b9/0x2380 > [ 3.116503] #2: ffffffff8f89fa20 (vmap_area_lock){+.+.}-{3:3}, at: > find_vmap_area+0x21/0x90 > [ 3.116503] irq event stamp: 571506 > [ 3.116503] hardirqs last enabled at (571505): > [<ffffffff8d422ff5>] __call_rcu_common+0x3a5/0xdc0 > [ 3.116503] hardirqs last disabled at (571506): > [<ffffffff900487f7>] rcu_torture_init+0x22b7/0x2380 > [ 3.116503] softirqs last enabled at (406916): > [<ffffffff8d2e91f2>] __local_bh_enable_ip+0x132/0x170 > [ 3.116503] softirqs last disabled at (406910): > [<ffffffff8e658e79>] inet_register_protosw+0x9/0x190 > [ 3.116503] Preemption disabled at: > [ 3.116503] [<ffffffff9004865a>] rcu_torture_init+0x211a/0x2380 > [ 3.116503] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G W > 6.5.0-rc4-rt2-yocto-preempt-rt+ #7 > [ 3.116503] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), > BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014 > [ 3.116503] Call Trace: > [ 3.116503] <TASK> > [ 3.116503] dump_stack_lvl+0x68/0xb0 > [ 3.116503] dump_stack+0x14/0x20 > [ 3.116503] __might_resched+0x221/0x2f0 > [ 3.116503] rt_spin_lock+0x60/0x130 > [ 3.116503] ? find_vmap_area+0x21/0x90 > [ 3.116503] find_vmap_area+0x21/0x90 > [ 3.116503] vmalloc_dump_obj+0x25/0xa0 > [ 3.116503] mem_dump_obj+0x22/0x90 > [ 3.116503] __call_rcu_common+0x920/0xdc0 > [ 3.116503] ? __pfx_rcu_torture_err_cb+0x10/0x10 > [ 3.116503] ? __pfx___call_rcu_common+0x10/0x10 > [ 3.116503] ? rcu_torture_init+0x22b7/0x2380 > [ 3.116503] ? debug_smp_processor_id+0x1b/0x30 > [ 3.116503] ? rcu_is_watching+0x3e/0xc0 > [ 3.116503] call_rcu_hurry+0x14/0x20 > [ 3.116503] rcu_torture_init+0x2186/0x2380 > [ 3.116503] ? __pfx_rcu_torture_init+0x10/0x10 > [ 3.116503] ? __pfx_rt_mutex_slowunlock+0x10/0x10 > [ 3.116503] ? __pfx_irq_debugfs_init+0x10/0x10 > [ 3.116503] ? __pfx_rcu_torture_leak_cb+0x10/0x10 > [ 3.116503] ? rt_spin_unlock+0x50/0xa0 > [ 3.116503] ? add_device_randomness+0xb1/0xe0 > [ 3.116503] ? __pfx_rcu_torture_leak_cb+0x10/0x10 > [ 3.116503] ? __pfx_rcu_torture_init+0x10/0x10 > [ 3.116503] do_one_initcall+0xd8/0x430 > [ 3.116503] ? __pfx_do_one_initcall+0x10/0x10 > [ 3.116503] ? __kasan_check_read+0x15/0x20 > [ 3.116503] kernel_init_freeable+0x3a0/0x670 > [ 3.116503] ? rt_spin_unlock+0x50/0xa0 > [ 3.116503] ? __pfx_kernel_init+0x10/0x10 > [ 3.116503] kernel_init+0x23/0x160 > [ 3.116503] ? __pfx_kernel_init+0x10/0x10 > [ 3.116503] ret_from_fork+0x40/0x60 > [ 3.116503] ? __pfx_kernel_init+0x10/0x10 > [ 3.116503] ret_from_fork_asm+0x1b/0x30 > [ 3.116503] </TASK> > [ 3.116503] non-slab/vmalloc memory > > > as described by Lei Zhen: > > v2 --> v3: > 1. I made statistics about the source of 'rhp'. kmem_valid_obj() accounts for > more than 97.5%, and vmalloc accounts for less than 1%. So change call > mem_dump_obj() to call kmem_dump_obj() can meet debugging requirements and > avoid the potential deadlock risk of vmalloc_dump_obj(). > - mem_dump_obj(rhp); > + if (kmem_valid_obj(rhp)) > + kmem_dump_obj(rhp); > > The discussion about vmap_area_lock deadlock in v2: > https://lkml.org/lkml/2022/11/11/493 > > so whether mem_dump_obj() should also be replaced by kmem_dump_obj() in > __call_rcu_common(), since the call_rcu() can be called in interrupt context, > this may issue recursive spin vmap_area_lock lock calls. > > Any thoughts ? or diff --git a/mm/util.c b/mm/util.c index ddfbb22dc187..fbc007adc108 100644 --- a/mm/util.c +++ b/mm/util.c @@ -1066,10 +1066,9 @@ void mem_dump_obj(void *object) if (kmem_dump_obj(object)) return; - if (vmalloc_dump_obj(object)) - return; - - if (virt_addr_valid(object)) + if (is_vmalloc_addr(object)) + type = "vmalloc memory"; + else if (virt_addr_valid(object)) type = "non-slab/vmalloc memory"; else if (object == NULL) type = "NULL pointer"; Thanks Zqiang > > Thanks > Zqiang