On 02/03/2016 01:57 PM, Sagi Grimberg wrote: > Hi Bart and Hannes, > >> This topic indeed needs further attention. I also encountered this >> challenge while adding scsi-mq support to the SRP initiator >> driver. What >> I learned while working on the SRP driver is the following: >> - Although I agree that requests and interrupts should be >> processed on >> the same processor (same physical chip) if the request has been >> submitted from the CPU closest to the HBA, I'm not convinced that >> processing request completions and interrupts on the same CPU core >> yields the best performance. I would appreciate it if there would >> remain some freedom in how to assign interrupts to CPU cores. > > This is true not only for this reason. Some block storage transports > (e.g. srp/iser) share the HBA with the networking stack and possibly > with user-space workloads in the case of RDMA. This is why I don't see > how would MSIX assignments can be done anywhere other than user-space. > Oh, I don't doubt that we should allow to have the MSIX assignments from user-space. But ATM block-mq assigned a fixed CPU <-> hardware queue mapping, and all we can do is to fixup things afterwards. Irrespective on whether it's 'best' for any given hardware. > However what I think we can do is have blk-mq ask the drivers > information about the MSIX mappings. This concept was introduced in > 2011 by Ben Hutchings with the CPU affinity reverse-mapping API [1]. > > Perhaps we'd want to have drivers provide blk-mq a struct cpu_rmap when > assigning the hctx mappings, or possibly per I/O if we want to be > agnostic to MSIX topology changes. I think this approach would solve > Hannes is experiencing. > > Thoughts? > Indeed, something like this. Quite some issues would be solved if we could push a hctx mapping into blk-mq, instead of having it assign its own made-up one. Cheers, Hannes -- Dr. Hannes Reinecke Teamlead Storage & Networking hare@xxxxxxx +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-block" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
