On Tue, Oct 25, 2022 at 10:32:28AM +0100, John Garry wrote: > On 25/10/2022 10:16, Ming Lei wrote: > > > > > I mentioned before that if no hctx->cpumask is online then we don't need > > > > > to allocate a request. That is because if no hctx->cpumask is online, > > > > > this means that original erroneous IO must be completed due to nature of > > > > > how blk-mq cpu hotplug handler works, i.e. drained, and then we don't > > > > > actually need to abort it any longer, so ok to not get a request. > > > > No, it is really not OK, if all cpus in hctx->cpumask are offline, you > > > > can't allocate > > > > request on the specified hw queue, then the erroneous IO can't be handled, > > > > then cpu hotplug handler may hang for ever. > > > If the erroneous IO is still in-flight from blk-mq perspective, then how can > > > hctx->cpumask still be offline? I thought that we guarantee that > > > hctx->cpumask cannot go offline until drained. > > Yeah, the draining is done before the cpu is offline. But the drain is > > simply waiting for the inflight IO to be completed. If the IO is failed > > during the waiting, you can't allocate such reserved request for error > > handling, then hang ever in blk_mq_hctx_notify_offline(). > > Actually if final cpu in hctx->cpumask is going offline, then hctx won't > queue any more requests, right? In this case I don't think we can queue on > that hctx anyway. I need to think about this more. It can be queued actually, but interrupt may not be delivered if managed irq is used. > > > > > If you just make it one driver private command, there can't be such > > issue. > > Well we're trying to use reserved requests for EH commands, which that goes > against. > > > Block layer is supposed for handling common case(normal io and pt io), > > I'd suggest to not put such special cases into block layer. > > It also supports reserved commands, which I would assume would be suitable > for EH scenarios. Then you have to be careful, as I mentioned, EH has to provide forward progress, if you let blk-mq allocate & submit EH request, the implied dependency from blk-mq has to be payed attention. Thanks, Ming
