[+intel-gfx]
Hello Jiangshan,
Hope you are doing well. I am Chaitanya from the linux graphics team in Intel.
This mail is regarding a regression we are seeing in our CI runs[1] on linux-next repository.
Since the version next-20240709 [2], we are seeing the following regression during boot up.
`````````````````````````````````````````````````````````````````````````````````
<4>[ 0.813758] ============================================
<4>[ 0.813758] WARNING: possible recursive locking detected
<4>[ 0.813758] 6.10.0-rc7-next-20240709-next-20240709-g82d01fe6ee52+ #1 Not tainted
<4>[ 0.813758] --------------------------------------------
<4>[ 0.813758] swapper/0/1 is trying to acquire lock:
<4>[ 0.813758] ffffffff8264adb0 (cpu_hotplug_lock){++++}-{0:0}, at: alloc_workqueue+0x31a/0x820
<4>[ 0.813758]
but task is already holding lock:
<4>[ 0.813758] ffffffff8264adb0 (cpu_hotplug_lock){++++}-{0:0}, at: padata_alloc+0x56/0x140
<4>[ 0.813758]
other info that might help us debug this:
<4>[ 0.813758] Possible unsafe locking scenario:
<4>[ 0.813758] CPU0
<4>[ 0.813758] ----
<4>[ 0.813758] lock(cpu_hotplug_lock);
<4>[ 0.813758] lock(cpu_hotplug_lock);
<4>[ 0.813758]
*** DEADLOCK ***
<4>[ 0.813758] May be due to missing lock nesting notation
<4>[ 0.813758] 1 lock held by swapper/0/1:
<4>[ 0.813758] #0: ffffffff8264adb0 (cpu_hotplug_lock){++++}-{0:0}, at: padata_alloc+0x56/0x140
<4>[ 0.813758]
stack backtrace:
<4>[ 0.813758] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.10.0-rc7-next-20240709-next-20240709-g82d01fe6ee52+ #1
<4>[ 0.813758] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4>[ 0.813758] Call Trace:
<4>[ 0.813758] <TASK>
<4>[ 0.813758] dump_stack_lvl+0x82/0xd0
<4>[ 0.813758] print_deadlock_bug+0x259/0x390
<4>[ 0.813758] __lock_acquire+0x733/0x1f60
<4>[ 0.813758] lock_acquire+0xd1/0x2f0
<4>[ 0.813758] ? alloc_workqueue+0x31a/0x820
<4>[ 0.813758] cpus_read_lock+0x3b/0xe0
<4>[ 0.813758] ? alloc_workqueue+0x31a/0x820
<4>[ 0.813758] alloc_workqueue+0x31a/0x820
<4>[ 0.813758] ? __pfx_pcrypt_init+0x10/0x10
<4>[ 0.813758] padata_alloc+0x6f/0x140
<4>[ 0.813758] pcrypt_init_padata+0x12/0x80
<4>[ 0.813758] ? __pfx_pcrypt_init+0x10/0x10
<4>[ 0.813758] pcrypt_init+0x3a/0xa0
<4>[ 0.813758] do_one_initcall+0x5c/0x2b0
<4>[ 0.813758] kernel_init_freeable+0x18e/0x340
<4>[ 0.813758] ? __pfx_kernel_init+0x10/0x10
<4>[ 0.813758] kernel_init+0x15/0x130
<4>[ 0.813758] ret_from_fork+0x2c/0x50
<4>[ 0.813758] ? __pfx_kernel_init+0x10/0x10
<4>[ 0.813758] ret_from_fork_asm+0x1a/0x30
<4>[ 0.813758] </TASK>
`````````````````````````````````````````````````````````````````````````````````
Details log can be found in [3].
After bisecting the tree, the following patch [4] seems to be the first "bad"
commit
`````````````````````````````````````````````````````````````````````````````````````````````````````````
commit 1726a17135905e2d2773f18d47bd4e17dd26e1ed
Author: Lai Jiangshan mailto:jiangshan.ljs@xxxxxxxxxxxx
Date: Thu Jul 4 11:49:13 2024 +0800
workqueue: Put PWQ allocation and WQ enlistment in the same lock C.S.
`````````````````````````````````````````````````````````````````````````````````````````````````````````
We could not revert the patch because of merge conflict but resetting to the parent of the commit seems to fix the issue
Could you please check why the patch causes this regression and provide a fix if necessary?
Thank you.
Regards
Chaitanya
[1] https://intel-gfx-ci.01.org/tree/linux-next/combined-alt.html?
[2] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20240709
[3] https://gfx-ci.igk.intel.com/tree/linux-next/next-20240709/bat-rpls-4/boot0.txt
[4] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20240709&id=1726a17135905e2d2773f18d47bd4e17dd26e1ed
