On Wed, May 27, 2020 at 11:09:53AM -0700, Alan Adamson wrote: > [ 139.895265] CPU: 5 PID: 2470 Comm: kworker/5:1H Not tainted 5.7.0-rc7+ #1 > [ 139.895266] Hardware name: Oracle Corporation ORACLE SERVER > X6-2/ASM,MOTHERBOARD,1U, BIOS 38050100 08/30/2016 > [ 139.895271] Workqueue: kblockd blk_mq_timeout_work > [ 139.895272] Call Trace: > [ 139.895279] dump_stack+0x6d/0x9a > [ 139.895281] should_fail.cold.5+0x32/0x42 > [ 139.895282] blk_should_fake_timeout+0x26/0x30 > [ 139.895283] blk_mq_complete_request+0x1b/0x120 > [ 139.895292] nvme_cancel_request+0x33/0x80 [nvme_core] > [ 139.895296] bt_tags_iter+0x48/0x50 > [ 139.895297] blk_mq_tagset_busy_iter+0x1eb/0x270 > [ 139.895299] ? nvme_try_sched_reset+0x40/0x40 [nvme_core] > [ 139.895301] ? nvme_try_sched_reset+0x40/0x40 [nvme_core] > [ 139.895305] nvme_dev_disable+0x2be/0x460 [nvme] > [ 139.895307] nvme_timeout.cold.80+0x9c/0x182 [nvme] > [ 139.895311] ? sched_clock+0x9/0x10 > [ 139.895315] ? sched_clock_cpu+0x11/0xc0 > [ 139.895320] ? __switch_to_asm+0x40/0x70 > [ 139.895321] blk_mq_check_expired+0x192/0x1b0 > [ 139.895322] bt_iter+0x52/0x60 > [ 139.895323] blk_mq_queue_tag_busy_iter+0x1a0/0x2e0 > [ 139.895325] ? __switch_to_asm+0x40/0x70 > [ 139.895326] ? __blk_mq_requeue_request+0xf0/0xf0 > [ 139.895326] ? __blk_mq_requeue_request+0xf0/0xf0 > [ 139.895329] ? compat_start_thread+0x20/0x40 > [ 139.895330] blk_mq_timeout_work+0x5a/0x130 > [ 139.895333] process_one_work+0x1ab/0x380 > [ 139.895334] worker_thread+0x37/0x3b0 > [ 139.895335] kthread+0x120/0x140 > [ 139.895337] ? create_worker+0x1b0/0x1b0 > [ 139.895337] ? kthread_park+0x90/0x90 > [ 139.895339] ret_from_fork+0x35/0x40 The driver reclaimed all outstanding tags and returned them to the block layer. This isn't faking a timeout anymore. The driver has done its part to reclaim lost commands. This is faking a broken block layer instead.