> -----Original Message----- > From: linux-scsi-owner@xxxxxxxxxxxxxxx [mailto:linux-scsi- > owner@xxxxxxxxxxxxxxx] On Behalf Of Jens Axboe > Sent: Thursday, November 10, 2016 10:18 PM > To: Hannes Reinecke; Christoph Hellwig > Cc: SCSI Mailing List; linux-block@xxxxxxxxxxxxxxx > Subject: Re: Reduced latency is killing performance > > On 11/10/2016 09:04 AM, Hannes Reinecke wrote: > > Hi all, > > > > this really feels like a follow-up to the discussion we've had in > > Santa Fe, but finally I'm able to substantiate it with some numbers. > > > > I've made a patch to enable the megaraid_sas driver for multiqueue. > > While this is pretty straightforward (I'll be sending the patchset > > later on), the results are ... interesting. > > > > I've run the 'ssd-test.fio' script from Jens' repository, and these > > results for MQ/SQ (- is mq, + is sq): > > > > Run status group 0 (all jobs): > > - READ: io=10641MB, aggrb=181503KB/s, minb=181503KB/s, > > maxb=181503KB/s, mint=60033msec, maxt=60033msec > > + READ: io=18370MB, aggrb=312572KB/s, minb=312572KB/s, > > maxb=312572KB/s, mint=60181msec, maxt=60181msec > > > > Run status group 1 (all jobs): > > - READ: io=441444KB, aggrb=7303KB/s, minb=7303KB/s, maxb=7303KB/s, > > mint=60443msec, maxt=60443msec > > + READ: io=223108KB, aggrb=3707KB/s, minb=3707KB/s, maxb=3707KB/s, > > mint=60182msec, maxt=60182msec > > > > Run status group 2 (all jobs): > > - WRITE: io=22485MB, aggrb=383729KB/s, minb=383729KB/s, > > maxb=383729KB/s, mint=60001msec, maxt=60001msec > > + WRITE: io=47421MB, aggrb=807581KB/s, minb=807581KB/s, > > maxb=807581KB/s, mint=60129msec, maxt=60129msec > > > > Run status group 3 (all jobs): > > - WRITE: io=489852KB, aggrb=8110KB/s, minb=8110KB/s, maxb=8110KB/s, > > mint=60399msec, maxt=60399msec > > + WRITE: io=489748KB, aggrb=8134KB/s, minb=8134KB/s, maxb=8134KB/s, > > mint=60207msec, maxt=60207msec > > > > Disk stats (read/write): > > - sda: ios=2834412/5878578, merge=0/0, ticks=86269292/48364836, > > in_queue=135345876, util=99.20% > > + sda: ios=205278/2680329, merge=4552593/9580622, > > ticks=12539912/12965228, in_queue=25512312, util=99.59% > > > > As you can see, we're really losing performance in the multiqueue case. > > And the main reason for that is that we submit about _10 times_ as > > much I/O as we do for the single-queue case. > > What's the setup like? I'm going to need more details. > > The baseline test is using the legacy path, single queue. The new test is > multiqueue, scsi-mq. What's sda? Hannes - Please share setup/config details so that I can also validate and post my findings. ` Kashyap > > > So I guess having an I/O scheduler is critical, even for the scsi-mq > > case. > > Each of these sections is a single job. For some reason we are not merging > as > well as we should, that's the reason for the performance loss. In fact, > we're not > merging at all. That's not IO scheduling. > > -- > Jens Axboe > > -- > To unsubscribe from this list: send the line "unsubscribe linux-scsi" in > the body of > a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-block" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
