On 2020-05-20 10:06, Christoph Hellwig wrote: > this series ensures I/O is quiesced before a cpu and thus the managed > interrupt handler is shut down. > > This patchset tries to address the issue by the following approach: > > - before the last cpu in hctx->cpumask is going to offline, mark this > hctx as inactive > > - disable preempt during allocating tag for request, and after tag is > allocated, check if this hctx is inactive. If yes, give up the > allocation and try remote allocation from online CPUs > > - before hctx becomes inactive, drain all allocated requests on this > hctx What is not clear to me is which assumptions about the relationship between interrupts and hardware queues this patch series is based on. Does this patch series perhaps only support a 1:1 mapping between interrupts and hardware queues? What if there are more hardware queues than interrupts? An example of a block driver that allocates multiple hardware queues is the NVMeOF initiator driver. From the NVMeOF initiator driver function nvme_rdma_alloc_tagset() and for the code that refers to I/O queues: set->nr_hw_queues = nctrl->queue_count - 1; >From nvme_rdma_alloc_io_queues(): nr_read_queues = min_t(unsigned int, ibdev->num_comp_vectors, min(opts->nr_io_queues, num_online_cpus())); nr_default_queues = min_t(unsigned int, ibdev->num_comp_vectors, min(opts->nr_write_queues, num_online_cpus())); nr_poll_queues = min(opts->nr_poll_queues, num_online_cpus()); nr_io_queues = nr_read_queues + nr_default_queues + nr_poll_queues; [ ... ] ctrl->ctrl.queue_count = nr_io_queues + 1; >From nvmf_parse_options(): /* Set defaults */ opts->nr_io_queues = num_online_cpus(); Can this e.g. result in 16 hardware queues being allocated for I/O even if the underlying RDMA adapter only supports four interrupt vectors? Does that mean that four hardware queues will be associated with each interrupt vector? If the CPU to which one of these interrupt vectors has been assigned is hotplugged, does that mean that four hardware queues have to be quiesced instead of only one as is done in patch 6/6? Thanks, Bart.
