On Tue, May 28, 2019 at 05:50:40PM +0100, John Garry wrote: > On 27/05/2019 16:02, Ming Lei wrote: > > Managed interrupts can not migrate affinity when their CPUs are offline. > > If the CPU is allowed to shutdown before they're returned, commands > > dispatched to managed queues won't be able to complete through their > > irq handlers. > > > > Wait in cpu hotplug handler until all inflight requests on the tags > > are completed or timeout. Wait once for each tags, so we can save time > > in case of shared tags. > > > > Based on the following patch from Keith, and use simple delay-spin > > instead. > > > > https://lore.kernel.org/linux-block/20190405215920.27085-1-keith.busch@xxxxxxxxx/ > > > > Some SCSI devices may have single blk_mq hw queue and multiple private > > completion queues, and wait until all requests on the private completion > > queue are completed. > > Hi Ming, > > I'm a bit concerned that this approach won't work due to ordering: it seems > that the IRQ would be shutdown prior to the CPU dead notification for the Managed IRQ shutdown is run in irq_migrate_all_off_this_cpu(), which is called in the callback of takedown_cpu(). And the CPU dead notification is always sent after that CPU becomes offline, see cpuhp_invoke_callback(). > last CPU in the mask (where we attempt to drain the queue associated with > the IRQ, which would require the IRQ to be still enabled). > > I hope that you can tell me that I'm wrong... Or you add one line printk in both irq_migrate_all_off_this_cpu() and blk_mq_hctx_notify_dead(), you will see if you are wrong. Thanks, Ming
