On Wed, 2017-11-01 at 12:08 +0800, Ming Lei wrote:
> On Wed, Nov 01, 2017 at 03:54:09AM +0000, Bart Van Assche wrote:
> > On Tue, 2017-10-31 at 09:47 +0800, Ming Lei wrote:
> > > On Mon, Oct 30, 2017 at 08:24:57PM +0000, Bart Van Assche wrote:
> > > > On Fri, 2017-10-27 at 13:38 +0800, Ming Lei wrote:
> > > > > On Fri, Oct 27, 2017 at 04:53:18AM +0000, Bart Van Assche wrote:
> > > > > > On Fri, 2017-10-27 at 12:43 +0800, Ming Lei wrote:
> > > > > > > The 1st patch removes the RESTART for TAG-SHARED because SCSI handles it
> > > > > > > by itself, and not necessary to waste CPU to do the expensive RESTART.
> > > > > > > And Roman Pen reported that this RESTART cuts half of IOPS in his case.
> > > > > > >
> > > > > > > The 2nd patch removes the RESTART when .get_budget returns BLK_STS_RESOURCE,
> > > > > > > and this RESTART is handled by SCSI's RESTART(scsi_end_request()) too.
> > > > > >
> > > > > > There are more block drivers than the SCSI core that share tags. If the
> > > > >
> > > > > Could you share us what the other in-tree driver which share tags is?
> > > >
> > > > I think the following in-tree drivers support shared tags (in alphabetical
> > > > order):
> > > > * null_blk. See also the shared_tags kernel module parameter.
> > > > * nvme. See also nvme_alloc_ns().
> > > > * scsi-mq.
> > >
> > > For both null_blk and nvme, we don't need to deal with cross-queue RESTART,
> > > because BLK_MQ_S_TAG_WAITING has handled it already.
> >
> > blk_mq_dispatch_wait_add() returns immediately if BLK_MQ_S_TAG_WAITING is
> > already set. Are you really sure that removing the restart mechanism doesn't
> > break the NVMe driver if there are multiple namespaces?>
> If this flag is set, that means blk_mq_dispatch_wake() will be run when
> driver tag is available, so the hw queue will be run at that time.
The only hw queue that will be rerun if BLK_MQ_S_TAG_WAITING is set is the hw
queue for which that flag was set. Sorry but I don't think that mechanism by
itself is sufficient to avoid queue lockups with the NVMe driver in case of
shared namespaces. See also the description of commit da55f2cc7841 ("blk-mq:
use sbq wait queues instead of restart for driver tags").
Bart.
