Hi,
在 2024/05/23 11:34, Yi Zhang 写道:
On Wed, May 22, 2024 at 4:39 PM Yu Kuai <yukuai1@xxxxxxxxxxxxxxx> wrote:
Hi,
在 2024/05/22 16:23, Li Nan 写道:
Hi, Yi Zhang
在 2024/5/22 12:40, Yi Zhang 写道:
Hello
With Sagi's new blktests case scenario[1], I tried the concurrent
power on/off for null-blk[2] and found it will lead to kernel
panic[3], please help check it and let me know if you need any
info/testing for it, thanks.
BTW, I will submit one blktests case for it later.
[1]
https://lore.kernel.org/linux-nvme/20240521085623.87681-1-sagi@xxxxxxxxxxx/
[2]Reproducer:
nullb1="/sys/kernel/config/nullb/nullb1"
modprobe null-blk nr_devices=0
mkdir $nullb1
echo 1024 > $nullb1/size
echo 1 > $nullb1/memory_backed
null_blk_power_loop() {
local iterations=10
for ((i = 1; i <= ${iterations}; i++)); do
echo 0 > $nullb1/power
echo 1 > $nullb1/power
done
}
null_blk_power_loop &
null_blk_power_loop &
I tried to fix this:
https://lore.kernel.org/all/20230610074618.2673673-1-yukuai1@xxxxxxxxxxxxxxx/
Hi, Yi, can you give the attached patch a test? I just rebase it with
block/for-next branch.
Hi Kuai
The issue cannot be reproduced now with your patch.
Thanks for the test. I'll resend the patch.
Kuai
Thanks,
Kuai
Concurrent creation and deletion of null_blk processes can lead to this
issue. Consider the following scenario:
T1 T2
echo 0 > power
null_del_dev
free nullb
echo 1 > power
null_add_dev
dev->nullb = nullb
...
dev->nullb = NULL
T2 creates a new device and sets dev->nullb, but T1 sets dev->nullb to NULL
later. after that, anyone trying to access dev->nullb will trigger NULL
pointer dereference.
I will fix it soon. Thanks for your report.
wait
[3]
[ 1200.017939] null_blk: disk nullb1 created
[ 1200.043860] BUG: kernel NULL pointer dereference, address:
0000000000000130
[ 1200.051627] #PF: supervisor write access in kernel mode
[ 1200.057458] #PF: error_code(0x0002) - not-present page
[ 1200.063192] PGD 0 P4D 0
[ 1200.066021] Oops: 0002 [#1] PREEMPT SMP PTI
[ 1200.070691] CPU: 0 PID: 1724 Comm: sh Not tainted
6.9.0-64.eln136.x86_64 #1
[ 1200.078462] Hardware name: Dell Inc. PowerEdge R730xd/ɲ�Pow, BIOS
2.19.0 12/12/2023
[ 1200.087105] RIP: 0010:_raw_spin_lock_irq+0x18/0x30
[ 1200.092459] Code: 66 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90
90 90 f3 0f 1e fa 0f 1f 44 00 00 fa 65 ff 05 47 28 26 7c 31 c0 ba 01
00 00 00 <f0> 0f b1 17 75 05 c3 cc cc cc cc 89 c6 e8 46 01 00 00 90 c3
cc cc
[ 1200.113416] RSP: 0018:ffffb973431e7420 EFLAGS: 00010046
[ 1200.119249] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000008
[ 1200.127212] RDX: 0000000000000001 RSI: 0000000000000000 RDI:
0000000000000130
[ 1200.135175] RBP: 0000000000000008 R08: ffff9645c44e4400 R09:
0000000000000000
[ 1200.143140] R10: 0000000000000014 R11: 0000000000000000 R12:
ffff964743a2d800
[ 1200.151103] R13: 0000000000000130 R14: ffff9645d05580f8 R15:
0000000000000001
[ 1200.159067] FS: 00007f67cea3b740(0000) GS:ffff964737a00000(0000)
knlGS:0000000000000000
[ 1200.168099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1200.174511] CR2: 0000000000000130 CR3: 00000002819c4001 CR4:
00000000001706f0
[ 1200.182473] Call Trace:
[ 1200.185200] <TASK>
[ 1200.187541] ? show_trace_log_lvl+0x1b0/0x2f0
[ 1200.192406] ? show_trace_log_lvl+0x1b0/0x2f0
[ 1200.197272] ? null_handle_rq+0x3f/0x500 [null_blk]
[ 1200.202735] ? __die_body.cold+0x8/0x12
[ 1200.207008] ? page_fault_oops+0x146/0x160
[ 1200.211583] ? exc_page_fault+0x73/0x160
[ 1200.215963] ? asm_exc_page_fault+0x26/0x30
[ 1200.220634] ? _raw_spin_lock_irq+0x18/0x30
[ 1200.225303] null_handle_rq+0x3f/0x500 [null_blk]
[ 1200.230567] ? pcpu_alloc+0x369/0x900
[ 1200.234654] ? lock_timer_base+0x76/0xa0
[ 1200.239035] null_process_cmd+0xb4/0x100 [null_blk]
[ 1200.244490] null_handle_cmd+0x36/0x130 [null_blk]
[ 1200.249849] null_queue_rq+0x130/0x200 [null_blk]
[ 1200.255110] ? __pfx_autoremove_wake_function+0x10/0x10
[ 1200.260948] __blk_mq_issue_directly+0x4b/0xc0
[ 1200.265911] blk_mq_try_issue_directly+0x88/0x110
[ 1200.271155] blk_mq_submit_bio+0x596/0x690
[ 1200.275720] submit_bio_noacct_nocheck+0x162/0x240
[ 1200.281071] ? submit_bio_noacct+0x24/0x540
[ 1200.285740] block_read_full_folio+0x1f8/0x300
[ 1200.290702] ? __pfx_blkdev_get_block+0x10/0x10
[ 1200.295760] ? __pfx_blkdev_read_folio+0x10/0x10
[ 1200.300913] ? __pfx_blkdev_read_folio+0x10/0x10
[ 1200.306064] filemap_read_folio+0x43/0xe0
[ 1200.310542] ? __pfx_blkdev_read_folio+0x10/0x10
[ 1200.315693] do_read_cache_folio+0x7c/0x190
[ 1200.320365] read_part_sector+0x33/0xb0
[ 1200.324650] read_lba+0xc1/0x180
[ 1200.328256] find_valid_gpt.constprop.0+0xe1/0x520
[ 1200.333609] efi_partition+0x7c/0x3a0
[ 1200.337699] ? snprintf+0x53/0x70
[ 1200.341401] ? __pfx_efi_partition+0x10/0x10
[ 1200.346170] check_partition+0x101/0x1c0
[ 1200.350550] bdev_disk_changed+0x193/0x330
[ 1200.355121] ? security_file_alloc+0x61/0xf0
[ 1200.359889] blkdev_get_whole+0x5f/0xa0
[ 1200.364170] bdev_open+0x205/0x3c0
[ 1200.367966] bdev_file_open_by_dev+0xbd/0x110
[ 1200.372829] disk_scan_partitions+0x6e/0x100
[ 1200.377594] device_add_disk+0x3bb/0x3c0
[ 1200.381972] null_add_dev+0x479/0x650 [null_blk]
[ 1200.387139] nullb_device_power_store+0x7c/0x120 [null_blk]
[ 1200.393370] configfs_write_iter+0xbc/0x120
[ 1200.398042] vfs_write+0x296/0x460
[ 1200.401841] ksys_write+0x6d/0xf0
[ 1200.405543] do_syscall_64+0x7e/0x160
[ 1200.409634] ? syscall_exit_work+0xf3/0x120
[ 1200.414302] ? syscall_exit_to_user_mode+0x71/0x1f0
[ 1200.419740] ? do_syscall_64+0x8a/0x160
[ 1200.424019] ? syscall_exit_work+0xf3/0x120
[ 1200.428686] ? syscall_exit_to_user_mode+0x71/0x1f0
[ 1200.434131] ? do_syscall_64+0x8a/0x160
[ 1200.438413] ? syscall_exit_work+0xf3/0x120
[ 1200.443081] ? syscall_exit_to_user_mode+0x71/0x1f0
[ 1200.448525] ? do_syscall_64+0x8a/0x160
[ 1200.452807] ? syscall_exit_work+0xf3/0x120
[ 1200.457475] ? syscall_exit_to_user_mode+0x71/0x1f0
[ 1200.462921] ? do_syscall_64+0x8a/0x160
[ 1200.467202] ? syscall_exit_to_user_mode+0x71/0x1f0
[ 1200.472638] ? do_syscall_64+0x8a/0x160
[ 1200.476918] ? exc_page_fault+0x73/0x160
[ 1200.481296] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[ 1200.486933] RIP: 0033:0x7f67ce8fda57
[ 1200.490931] Code: 0f 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7
0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00
00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89
74 24
[ 1200.511886] RSP: 002b:00007ffd30172638 EFLAGS: 00000246 ORIG_RAX:
0000000000000001
[ 1200.520335] RAX: ffffffffffffffda RBX: 0000000000000002 RCX:
00007f67ce8fda57
[ 1200.528297] RDX: 0000000000000002 RSI: 0000557fdb881c90 RDI:
0000000000000001
[ 1200.536258] RBP: 0000557fdb881c90 R08: 0000000000000000 R09:
00007f67ce9b14e0
[ 1200.544219] R10: 00007f67ce9b13e0 R11: 0000000000000246 R12:
0000000000000002
[ 1200.552181] R13: 00007f67ce9fb780 R14: 0000000000000002 R15:
00007f67ce9f69e0
[ 1200.560145] </TASK>
[ 1200.562581] Modules linked in: null_blk sunrpc vfat fat
intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal
intel_powerclamp coretemp dell_wmi_descriptor kvm_intel sparse_keymap
cdc_ether rfkill ipmi_ssif usbnet video kvm mei_me iTCO_wdt dcdbas mii
iTCO_vendor_support mei ipmi_si rapl mgag200 mxm_wmi pcspkr
intel_cstate ipmi_devintf acpi_power_meter ipmi_msghandler
intel_uncore i2c_algo_bit lpc_ich fuse xfs sd_mod sg nvme ahci libahci
crct10dif_pclmul crc32_pclmul crc32c_intel libata ghash_clmulni_intel
nvme_core tg3 megaraid_sas nvme_auth t10_pi wmi dm_mirror
dm_region_hash dm_log dm_mod [last unloaded: null_blk]
[ 1200.624314] CR2: 0000000000000130
[ 1200.628012] ---[ end trace 0000000000000000 ]---
[ 1200.688370] RIP: 0010:_raw_spin_lock_irq+0x18/0x30
[ 1200.693747] Code: 66 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90
90 90 f3 0f 1e fa 0f 1f 44 00 00 fa 65 ff 05 47 28 26 7c 31 c0 ba 01
00 00 00 <f0> 0f b1 17 75 05 c3 cc cc cc cc 89 c6 e8 46 01 00 00 90 c3
cc cc
[ 1200.714704] RSP: 0018:ffffb973431e7420 EFLAGS: 00010046
[ 1200.720535] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000008
[ 1200.728498] RDX: 0000000000000001 RSI: 0000000000000000 RDI:
0000000000000130
[ 1200.736461] RBP: 0000000000000008 R08: ffff9645c44e4400 R09:
0000000000000000
[ 1200.744423] R10: 0000000000000014 R11: 0000000000000000 R12:
ffff964743a2d800
[ 1200.752384] R13: 0000000000000130 R14: ffff9645d05580f8 R15:
0000000000000001
[ 1200.760348] FS: 00007f67cea3b740(0000) GS:ffff964737a00000(0000)
knlGS:0000000000000000
[ 1200.769378] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1200.775789] CR2: 0000000000000130 CR3: 00000002819c4001 CR4:
00000000001706f0
[ 1200.783754] Kernel panic - not syncing: Fatal exception
[ 1200.789636] Kernel Offset: 0x2000000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 1200.856453] ---[ end Kernel panic - not syncing: Fatal exception ]---
Best Regards,
Yi Zhang
.
