> -----Original Message----- > From: Hannes Reinecke [mailto:hare@xxxxxxx] > Sent: Monday, February 5, 2018 12:28 PM > To: Ming Lei; Jens Axboe; linux-block@xxxxxxxxxxxxxxx; Christoph Hellwig; > Mike Snitzer > Cc: linux-scsi@xxxxxxxxxxxxxxx; Arun Easi; Omar Sandoval; Martin K . > Petersen; > James Bottomley; Christoph Hellwig; Don Brace; Kashyap Desai; Peter > Rivera; > Paolo Bonzini; Laurence Oberman > Subject: Re: [PATCH 0/5] blk-mq/scsi-mq: support global tags & introduce > force_blk_mq > > On 02/03/2018 05:21 AM, Ming Lei wrote: > > Hi All, > > > > This patchset supports global tags which was started by Hannes > > originally: > > > > https://marc.info/?l=linux-block&m=149132580511346&w=2 > > > > Also inroduce 'force_blk_mq' to 'struct scsi_host_template', so that > > driver can avoid to support two IO paths(legacy and blk-mq), > > especially recent discusion mentioned that SCSI_MQ will be enabled at > default soon. > > > > https://marc.info/?l=linux-scsi&m=151727684915589&w=2 > > > > With the above two changes, it should be easier to convert SCSI drivers' > > reply queue into blk-mq's hctx, then the automatic irq affinity issue > > can be solved easily, please see detailed descrption in commit log. > > > > Also drivers may require to complete request on the submission CPU for > > avoiding hard/soft deadlock, which can be done easily with blk_mq too. > > > > https://marc.info/?t=151601851400001&r=1&w=2 > > > > The final patch uses the introduced 'force_blk_mq' to fix virtio_scsi > > so that IO hang issue can be avoided inside legacy IO path, this issue > > is a bit generic, at least HPSA/virtio-scsi are found broken with > > v4.15+. > > > > Thanks > > Ming > > > > Ming Lei (5): > > blk-mq: tags: define several fields of tags as pointer > > blk-mq: introduce BLK_MQ_F_GLOBAL_TAGS > > block: null_blk: introduce module parameter of 'g_global_tags' > > scsi: introduce force_blk_mq > > scsi: virtio_scsi: fix IO hang by irq vector automatic affinity > > > > block/bfq-iosched.c | 4 +-- > > block/blk-mq-debugfs.c | 11 ++++---- > > block/blk-mq-sched.c | 2 +- > > block/blk-mq-tag.c | 67 > > +++++++++++++++++++++++++++++----------------- > > block/blk-mq-tag.h | 15 ++++++++--- > > block/blk-mq.c | 31 ++++++++++++++++----- > > block/blk-mq.h | 3 ++- > > block/kyber-iosched.c | 2 +- > > drivers/block/null_blk.c | 6 +++++ > > drivers/scsi/hosts.c | 1 + > > drivers/scsi/virtio_scsi.c | 59 > > +++------------------------------------- > > include/linux/blk-mq.h | 2 ++ > > include/scsi/scsi_host.h | 3 +++ > > 13 files changed, 105 insertions(+), 101 deletions(-) > > > Thanks Ming for picking this up. > > I'll give it a shot and see how it behaves on other hardware. Ming - There is no way we can enable global tags from SCSI stack in this patch series. I still think we have no solution for issue described below in this patch series. https://marc.info/?t=151601851400001&r=1&w=2 What we will be doing is just use global tag HBA wide instead of h/w queue based. We still have more than one reply queue ending up completion one CPU. Try to reduce MSI-x vector of megaraid_sas or mpt3sas driver via module parameter to simulate the issue. We need more number of Online CPU than reply-queue. We may see completion redirected to original CPU because of "QUEUE_FLAG_SAME_FORCE", but ISR of low level driver can keep one CPU busy in local ISR routine. Kashyap > > Cheers, > > Hannes > -- > Dr. Hannes Reinecke Teamlead Storage & Networking > hare@xxxxxxx +49 911 74053 688 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 > (AG Nürnberg)
