On 29/10/2019 10:05, Ming Lei wrote:
But this patch does wait for completion of in-flight request before
shutdown the last CPU of this hctx.
Hi Ming,
It may actually be a request from a hctx which is not shut down which errors
and causes the time out. I'm still checking.
If that is the case, blk_mq_hctx_drain_inflight_rqs() will wait for
completion of this request.
The only chance it is missed is that the last CPU of this hctx becomes
offline just when this request stays in request list after it is
retried from EH.
BTW, Can you let me know exactly where you want the debug for "Or
blk_mq_hctx_next_cpu() may still run WORK_CPU_UNBOUND schedule after
all CPUs are offline, could you add debug message in that branch?"
You can add the following debug message, then reproduce the issue and
see if the debug log is dumped.
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 06081966549f..5a98a7b79c0d 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -1452,6 +1452,10 @@ static int blk_mq_hctx_next_cpu(struct blk_mq_hw_ctx *hctx)
*/
hctx->next_cpu = next_cpu;
hctx->next_cpu_batch = 1;
+
+ printk(KERN_WARNING "CPU %d, schedule from (dead) hctx %s\n",
+ raw_smp_processor_id(),
+ cpumask_empty(hctx->cpumask) ? "inactive": "active");
return WORK_CPU_UNBOUND;
We don't seem to be hitting this.
So the error generally happens after the CPUs have been hot unplugged
for some time. I find a SCSI IO errors in the LLDD, having been run on
an up hctx - blk-mq requeues it on same hctx, but it only dispatches
eventually after some other SCSI IO timesout - I assume that IO timesout
due to earlier scsi_set_blocked() call.
Continuing to look....
Thanks,
John
}
Thanks,
Ming
.
