On Fri, Mar 22, 2024 at 10:40:14AM -0700, Bart Van Assche wrote:
> From: Chengming Zhou <zhouchengming@xxxxxxxxxxxxx>
>
> commit e5c0ca13659e9d18f53368d651ed7e6e433ec1cf upstream.
>
> Chuck reported [1] an IO hang problem on NFS exports that reside on SATA
> devices and bisected to commit 615939a2ae73 ("blk-mq: defer to the normal
> submission path for post-flush requests").
>
> We analysed the IO hang problem, found there are two postflush requests
> waiting for each other.
>
> The first postflush request completed the REQ_FSEQ_DATA sequence, so go to
> the REQ_FSEQ_POSTFLUSH sequence and added in the flush pending list, but
> failed to blk_kick_flush() because of the second postflush request which
> is inflight waiting in scheduler queue.
>
> The second postflush waiting in scheduler queue can't be dispatched because
> the first postflush hasn't released scheduler resource even though it has
> completed by itself.
>
> Fix it by releasing scheduler resource when the first postflush request
> completed, so the second postflush can be dispatched and completed, then
> make blk_kick_flush() succeed.
>
> While at it, remove the check for e->ops.finish_request, as all
> schedulers set that. Reaffirm this requirement by adding a WARN_ON_ONCE()
> at scheduler registration time, just like we do for insert_requests and
> dispatch_request.
>
> [1] https://lore.kernel.org/all/7A57C7AE-A51A-4254-888B-FE15CA21F9E9@xxxxxxxxxx/
>
> Link: https://lore.kernel.org/linux-block/20230819031206.2744005-1-chengming.zhou@xxxxxxxxx/
> Reported-by: kernel test robot <oliver.sang@xxxxxxxxx>
> Closes: https://lore.kernel.org/oe-lkp/202308172100.8ce4b853-oliver.sang@xxxxxxxxx
> Fixes: 615939a2ae73 ("blk-mq: defer to the normal submission path for post-flush requests")
> Reported-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
> Signed-off-by: Chengming Zhou <zhouchengming@xxxxxxxxxxxxx>
> Tested-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
> Link: https://lore.kernel.org/r/20230813152325.3017343-1-chengming.zhou@xxxxxxxxx
> [axboe: folded in incremental fix and added tags]
> Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
> [bvanassche: changed RQF_USE_SCHED into RQF_ELVPRIV; restored the
> finish_request pointer check before calling finish_request and removed
> the new warning from the elevator code. This patch fixes an I/O hang
> when submitting a REQ_FUA request to a request queue for a zoned block
> device for which FUA has been disabled (QUEUE_FLAG_FUA is not set).]
> Signed-off-by: Bart Van Assche <bvanassche@xxxxxxx>
> ---
> block/blk-mq.c | 24 +++++++++++++++++++++---
> 1 file changed, 21 insertions(+), 3 deletions(-)
Now queued up, thanks.
greg k-h
