On Wed 07-01-15 09:22:13, Lee Duncan wrote: > On 01/07/2015 08:25 AM, Sagi Grimberg wrote: > > Hi everyone, > > > > Now that scsi-mq is fully included, we need an iSCSI initiator that > > would use it to achieve scalable performance. The need is even greater > > for iSCSI offload devices and transports that support multiple HW > > queues. As iSER maintainer I'd like to discuss the way we would choose > > to implement that in iSCSI. > > > > My measurements show that iSER initiator can scale up to ~2.1M IOPs > > with multiple sessions but only ~630K IOPs with a single session where > > the most significant bottleneck the (single) core processing > > completions. > > > > In the existing single connection per session model, given that command > > ordering must be preserved session-wide, we end up in a serial command > > execution over a single connection which is basically a single queue > > model. The best fit seems to be plugging iSCSI MCS as a multi-queued > > scsi LLDD. In this model, a hardware context will have a 1x1 mapping > > with an iSCSI connection (TCP socket or a HW queue). > > > > iSCSI MCS and it's role in the presence of dm-multipath layer was > > discussed several times in the past decade(s). The basic need for MCS is > > implementing a multi-queue data path, so perhaps we may want to avoid > > doing any type link aggregation or load balancing to not overlap > > dm-multipath. For example we can implement ERL=0 (which is basically the > > scsi-mq ERL) and/or restrict a session to a single portal. > > > > As I see it, the todo's are: > > 1. Getting MCS to work (kernel + user-space) with ERL=0 and a > > round-robin connection selection (per scsi command execution). > > 2. Plug into scsi-mq - exposing num_connections as nr_hw_queues and > > using blk-mq based queue (conn) selection. > > 3. Rework iSCSI core locking scheme to avoid session-wide locking > > as much as possible. > > 4. Use blk-mq pre-allocation and tagging facilities. > > > > I've recently started looking into this. I would like the community to > > agree (or debate) on this scheme and also talk about implementation > > with anyone who is also interested in this. > > > > Cheers, > > Sagi. > > I started looking at this last year (and Hannes' suggestion), and would > love to join the discussion. > > Please add me to the list of those that wish to attend. For that please send a separate email with attend request as described in the call for proposals. Thanks! Honza -- Jan Kara <jack@xxxxxxx> SUSE Labs, CR -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
