On Wed, Nov 10, 2021 at 10:15 AM Vishal Verma <vverma@xxxxxxxxxxxxxxxx> wrote:
>
> commit 021a24460dc2 ("block: add QUEUE_FLAG_NOWAIT") added support
> for checking whether a given bdev supports handling of REQ_NOWAIT or not.
> Since then commit 6abc49468eea ("dm: add support for REQ_NOWAIT and enable
> it for linear target") added support for REQ_NOWAIT for dm. This uses
> a similar approach to incorporate REQ_NOWAIT for md based bios.
>
> This patch was tested using t/io_uring tool within FIO. A nvme drive
> was partitioned into 2 partitions and a simple raid 0 configuration
> /dev/md0 was created.
>
> md0 : active raid0 nvme4n1p1[1] nvme4n1p2[0]
> 937423872 blocks super 1.2 512k chunks
>
> Before patch:
>
> $ ./t/io_uring /dev/md0 -p 0 -a 0 -d 1 -r 100
>
> Running top while the above runs:
>
> $ ps -eL | grep $(pidof io_uring)
>
> 38396 38396 pts/2 00:00:00 io_uring
> 38396 38397 pts/2 00:00:15 io_uring
> 38396 38398 pts/2 00:00:13 iou-wrk-38397
>
> We can see iou-wrk-38397 io worker thread created which gets created
> when io_uring sees that the underlying device (/dev/md0 in this case)
> doesn't support nowait.
>
> After patch:
>
> $ ./t/io_uring /dev/md0 -p 0 -a 0 -d 1 -r 100
>
> Running top while the above runs:
>
> $ ps -eL | grep $(pidof io_uring)
>
> 38341 38341 pts/2 00:10:22 io_uring
> 38341 38342 pts/2 00:10:37 io_uring
>
> After running this patch, we don't see any io worker thread
> being created which indicated that io_uring saw that the
> underlying device does support nowait. This is the exact behaviour
> noticed on a dm device which also supports nowait.
>
> For all the other raid personalities except raid0, we would need
> to train pieces which involves make_request fn in order for them
> to correctly handle REQ_NOWAIT.
>
> Signed-off-by: Vishal Verma <vverma@xxxxxxxxxxxxxxxx>
> ---
> drivers/md/md.c | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 5111ed966947..a30c78afcab6 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -419,6 +419,11 @@ void md_handle_request(struct mddev *mddev, struct bio *bio)
> if (is_suspended(mddev, bio)) {
> DEFINE_WAIT(__wait);
> for (;;) {
> + /* Bail out if REQ_NOWAIT is set for the bio */
> + if (bio->bi_opf & REQ_NOWAIT) {
> + bio_wouldblock_error(bio);
> + return;
> + }
We need a rcu_read_unlock() before bio_wouldbock_error(bio).
> prepare_to_wait(&mddev->sb_wait, &__wait,
> TASK_UNINTERRUPTIBLE);
> if (!is_suspended(mddev, bio))
> @@ -5792,6 +5797,7 @@ int md_run(struct mddev *mddev)
> int err;
> struct md_rdev *rdev;
> struct md_personality *pers;
> + bool nowait = true;
>
> if (list_empty(&mddev->disks))
> /* cannot run an array with no devices.. */
> @@ -5862,8 +5868,13 @@ int md_run(struct mddev *mddev)
> }
> }
> sysfs_notify_dirent_safe(rdev->sysfs_state);
> + nowait = nowait && blk_queue_nowait(bdev_get_queue(rdev->bdev));
> }
>
> + /* Set the NOWAIT flags if all underlying devices support it */
> + if (nowait)
> + blk_queue_flag_set(QUEUE_FLAG_NOWAIT, mddev->queue);
> +
> if (!bioset_initialized(&mddev->bio_set)) {
> err = bioset_init(&mddev->bio_set, BIO_POOL_SIZE, 0, BIOSET_NEED_BVECS);
> if (err)
> @@ -7007,6 +7018,15 @@ static int hot_add_disk(struct mddev *mddev, dev_t dev)
> set_bit(MD_SB_CHANGE_DEVS, &mddev->sb_flags);
> if (!mddev->thread)
> md_update_sb(mddev, 1);
> + /*
> + * If the new disk does not support REQ_NOWAIT,
> + * disable on the whole MD.
> + */
> + if (!blk_queue_nowait(bdev_get_queue(rdev->bdev))) {
> + pr_info("%s: Disabling nowait because %s does not support nowait\n",
> + mdname(mddev), bdevname(rdev->bdev, b));
> + blk_queue_flag_clear(QUEUE_FLAG_NOWAIT, mddev->queue);
> + }
> /*
> * Kick recovery, maybe this spare has to be added to the
> * array immediately.
> --
> 2.17.1
>
