Hi Chaitanya Sorry for the late response, as it take me some time to bisect this issue, seems it was introduced with the below commit: 23f3e3272e7a4d9fb870485cd6df1e4f9539282c Author: Xiao Ni <xni@xxxxxxxxxx> Date: Thu Feb 9 11:19:30 2023 +0800 block: Merge bio before checking ->cached_rq Here are the steps I used to reproduce it: 1. do fio stress test on three nvme disks background 2. offline the first three cpus # echo 0 > /sys/devices/system/cpu/cpu1/online # echo 0 > /sys/devices/system/cpu/cpu2/online # echo 0 > /sys/devices/system/cpu/cpu3/online 3. update each disk's scheduler/nr_requests for disk in nvme0n1 nvme1n1 nvme2n1; do { scheds="mq-deadline kyber bfq" for sched in $scheds; do echo $sched > /sys/block/$disk/queue/scheduler max_nr="$(cat /sys/block/$disk/queue/nr_requests)" for ((nr = 4; nr <= max_nr; nr++)); do echo $nr > /sys/block/$disk/queue/nr_requests done done } & done 4. online the cpus echo 1 > /sys/devices/system/cpu/cpu1/online echo 1 > /sys/devices/system/cpu/cpu2/online echo 1 > /sys/devices/system/cpu/cpu3/online On Sun, Apr 16, 2023 at 12:52 PM Chaitanya Kulkarni <chaitanyak@xxxxxxxxxx> wrote: > > On 4/14/23 08:04, Yi Zhang wrote: > > Hello > > > > Bellow panic triggered during nvme disks queue/nr_request update with > > fio background, pls help check it and let me know if you need any > > info/test, thanks. > > > > > > > Is it by any specific blktests ? > > can you please share the steps ? > > -ck > > -- Best Regards, Yi Zhang