> -----Original Message----- > From: Hannes Reinecke [mailto:hare@xxxxxxx] > Sent: Tuesday, January 31, 2017 4:47 PM > To: Christoph Hellwig > Cc: Martin K. Petersen; James Bottomley; linux-scsi@xxxxxxxxxxxxxxx; Sathya > Prakash; Kashyap Desai; mpt-fusionlinux.pdl@xxxxxxxxxxxx > Subject: Re: [PATCH 00/10] mpt3sas: full mq support > > On 01/31/2017 11:02 AM, Christoph Hellwig wrote: > > On Tue, Jan 31, 2017 at 10:25:50AM +0100, Hannes Reinecke wrote: > >> Hi all, > >> > >> this is a patchset to enable full multiqueue support for the mpt3sas > driver. > >> While the HBA only has a single mailbox register for submitting > >> commands, it does have individual receive queues per MSI-X interrupt > >> and as such does benefit from converting it to full multiqueue support. > > > > Explanation and numbers on why this would be beneficial, please. > > We should not need multiple submissions queues for a single register > > to benefit from multiple completion queues. > > > Well, the actual throughput very strongly depends on the blk-mq-sched > patches from Jens. > As this is barely finished I didn't post any numbers yet. > > However: > With multiqueue support: > 4k seq read : io=60573MB, bw=1009.2MB/s, iops=258353, runt= 60021msec > With scsi-mq on 1 queue: > 4k seq read : io=17369MB, bw=296291KB/s, iops=74072, runt= 60028msec So > yes, there _is_ a benefit. > > (Which is actually quite cool, as these tests were done on a SAS3 HBA, so > we're getting close to the theoretical maximum of 1.2GB/s). > (Unlike the single-queue case :-) Hannes - Can you share detail about setup ? How many drives do you have and how is connection (enclosure -> drives. ??) ? To me it looks like current mpt3sas driver might be taking more hit in spinlock operation (penalty on NUMA arch is more compare to single core server) unlike we have in megaraid_sas driver use of shared blk tag. I mean " [PATCH 08/10] mpt3sas: lockless command submission for scsi-mq" patch is improving performance removing spinlock overhead and attempting to get request using blk_tags. Are you seeing performance improvement if you hard code nr_hw_queues = 1 in below code changes part of "[PATCH 10/10] mpt3sas: scsi-mq interrupt steering" @@ -9054,6 +9071,8 @@ static void sas_device_make_active(struct MPT3SAS_ADAPTER *ioc, shost->max_lun = max_lun; shost->transportt = mpt3sas_transport_template; shost->unique_id = ioc->id; + if (shost->use_blk_mq) + shost->nr_hw_queues = num_online_cpus(); Thanks, Kashyap > > Cheers, > > Hannes > -- > Dr. Hannes Reinecke Teamlead Storage & Networking > hare@xxxxxxx +49 911 74053 688 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 > (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
