Re: [blk-mq Bug] race on removing hctx->dispatch_wait from wait queue

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, 2018-06-24 at 18:16 +0800, Ming Lei wrote:
> Now I am revisiting 'TAG_SHARED in restart' again for the long delay issue
> of SCSI LUN probe. And found there is one bug in blk_mq_mark_tag_wait():
> 
> - hctx->dispatch_wait is added to wait queue by holding hctx->lock and
> the wait queue's lock
> 
> - no hctx->lock is held when removing hctx->dispatch_wait from wait
>   queue.
> 
> - so the two actions(add, remove) may happen meantime since
>   hctx->dispatch_wait can be added to different wait queues.
> 
> Turns out this issue can be observed easily by applying the patches[2],
> which is for removing 'TAG_SHARED in restart', then run simple shared-tag
> null_blk test[4].
> 
> But if the hctx->lock is held in blk_mq_dispatch_wake(), as done in
> patch [3], there isn't such issue at all, so it shows this issue is
> related with hctx->lock, and adding/removing hctx->dispatch_wait to
> wait queue. But the way of holding hctx->lock in irq context may not
> be one accepted solution, since it has been avoided from the beginning
> of blk-mq.
> 
> So does anyone have better ideas for this issue?
> 
> So far, follows what I thought of:
> 
> 1) fix the mechanism of blk_mq_mark_tag_wait(), and removing
> 'TAG_SHARED in restart', then we can fix the long delay issue of
> SCSI LUN probe, meantime performance can got improved, as I observed,
> this way may improve IOPS by 20~30% in multi-LUN scsi_debug test.
> But the issue is how to fix?
> 
> 2) keep 'TAG_SHARED in restart' and let it cover the issue of
> blk_mq_mark_tag_wait() as now, then try to improve 'TAG_SHARED in restart'
> in another way, so that performance can be better, and synchronize_rcu()
> can be removed from blk_mq_del_queue_tag_set(), then SCSI LUN probe long
> delay can be fixed. I had wrote patches to do that last year. If anyone
> is interested, I may post it out.
> 
> Or other ideas, any comments & ideas are welcome!

Please have a look at [PATCH] blk-mq: Fix a race condition in blk_mq_mark_tag_wait(),
16 Jan 2018 (https://www.mail-archive.com/linux-block@xxxxxxxxxxxxxxx/msg17474.html).

Thanks,

Bart.








[Index of Archives]     [Linux RAID]     [Linux SCSI]     [Linux ATA RAID]     [IDE]     [Linux Wireless]     [Linux Kernel]     [ATH6KL]     [Linux Bluetooth]     [Linux Netdev]     [Kernel Newbies]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Device Mapper]

  Powered by Linux