Hi Guys, For minimizing IO latency, blk-mq often completes IO in the hardware interrupt context. However, in case of multiple HBAs, or multiple storage devices attached to same HBA, if one CPU core has to handle interrupts of IOs submitted from multile CPU , there is risk to lock up CPUs[1][2]. Follows the idea when there is mutliple IO submitter and single CPU for completing these IOs: 1) in case of multiple storages attached to one single HBA, these storages may handle IO more quickly than single CPU core 2) in case of multiple HBAs in one system, one single effective CPU can be selected for handling interrupts from several queues, then multiple storages still may handle IO more quickly than single CPU core. When handling IO completion in interrupt context, IO latency is good, but there is risk to lock up CPU. When moving IO completion to process context via NAPI or threaded interrupt handler, CPU lockup can be avoided. So people try to move part of IO completion into process context for avoiding CPU lockup, meantime trying to not hurt IO performance, such as Keith's work[3]. However, it is hard to partition IO completion in the two contexts, if less work is moved to process context, risk of locking up CPU can't be eliminated; if more work is moved to process context, extra IO latency is introduced, then IO performance is hurt. Interrupt flood information can be one useful hint for partitioning IO completion work into the two contexts effectively. The 1st two patches implement interrpupt flood detection, and the 3rd patch applies the hint to complet IO in process context when interrupt flood happens. This way avoids CPU lockup, meantime IO performance isn't hurt obviously. [1] https://lore.kernel.org/lkml/1566281669-48212-1-git-send-email-longli@xxxxxxxxxxxxxxxxx/ [2] https://lore.kernel.org/lkml/a7ef3810-31af-013a-6d18-ceb6154aa2ef@xxxxxxxxxx/ [3] https://lore.kernel.org/linux-nvme/20191209175622.1964-1-kbusch@xxxxxxxxxx/T/#t Ming Lei (3): sched/core: add API for exporting runqueue clock softirq: implement interrupt flood detection blk-mq: complete request in rescuer process context in case of irq flood block/blk-mq.c | 68 ++++++++++++++++- drivers/base/cpu.c | 23 ++++++ include/linux/hardirq.h | 2 + include/linux/sched.h | 2 + kernel/sched/core.c | 5 ++ kernel/softirq.c | 161 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 260 insertions(+), 1 deletion(-) Cc: Long Li <longli@xxxxxxxxxxxxx> Cc: Ingo Molnar <mingo@xxxxxxxxxx>, Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx> Cc: Keith Busch <keith.busch@xxxxxxxxx> Cc: Christoph Hellwig <hch@xxxxxx> Cc: Sagi Grimberg <sagi@xxxxxxxxxxx> Cc: John Garry <john.garry@xxxxxxxxxx> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx> Cc: Hannes Reinecke <hare@xxxxxxxx> -- 2.20.1
