Re: [LSF/MM TOPIC] multiqueue and interrupt assignment

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Bart and Hannes,

This topic indeed needs further attention. I also encountered this
challenge while adding scsi-mq support to the SRP initiator driver. What
I learned while working on the SRP driver is the following:
- Although I agree that requests and interrupts should be processed on
   the same processor (same physical chip) if the request has been
   submitted from the CPU closest to the HBA, I'm not convinced that
   processing request completions and interrupts on the same CPU core
   yields the best performance. I would appreciate it if there would
   remain some freedom in how to assign interrupts to CPU cores.

This is true not only for this reason. Some block storage transports
(e.g. srp/iser) share the HBA with the networking stack and possibly
with user-space workloads in the case of RDMA. This is why I don't see
how would MSIX assignments can be done anywhere other than user-space.

However what I think we can do is have blk-mq ask the drivers
information about the MSIX mappings. This concept was introduced in
2011 by Ben Hutchings with the CPU affinity reverse-mapping API [1].

Perhaps we'd want to have drivers provide blk-mq a struct cpu_rmap when
assigning the hctx mappings, or possibly per I/O if we want to be
agnostic to MSIX topology changes. I think this approach would solve
Hannes is experiencing.

Thoughts?

[1]:
commit c39649c331c70952700f99832b03f87e9d7f5b4b
Author: Ben Hutchings <bhutchings@xxxxxxxxxxxxxx>
Date:   Wed Jan 19 11:03:25 2011 +0000

    lib: cpu_rmap: CPU affinity reverse-mapping

    When initiating I/O on a multiqueue and multi-IRQ device, we may want
    to select a queue for which the response will be handled on the same
    or a nearby CPU.  This requires a reverse-map of IRQ affinity.  Add
    library functions to support a generic reverse-mapping from CPUs to
    objects with affinity and the specific case where the objects are
    IRQs.

    Signed-off-by: Ben Hutchings <bhutchings@xxxxxxxxxxxxxx>
    Signed-off-by: David S. Miller <davem@xxxxxxxxxxxxx>
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [SCSI Target Devel]     [Linux SCSI Target Infrastructure]     [Kernel Newbies]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Linux IIO]     [Samba]     [Device Mapper]
  Powered by Linux