On Wed, Mar 03, 2021 at 07:57:38PM +0800, Jeffle Xu wrote:
> Offer one fastpath for bio-based polling when bio submitted to dm
> device is not split.
>
> In this case, there will be only one bio submitted to only one polling
> hw queue of one underlying mq device, and thus we don't need to track
> all split bios or iterate through all polling hw queues. The pointer to
> the polling hw queue the bio submitted to is returned here as the
> returned cookie. In this case, the polling routine will call
> mq_ops->poll() directly with the hw queue converted from the input
> cookie.
>
> If the original bio submitted to dm device is split to multiple bios and
> thus submitted to multiple polling hw queues, the polling routine will
> fall back to iterating all hw queues (in polling mode) of all underlying
> mq devices.
>
> Signed-off-by: Jeffle Xu <jefflexu@xxxxxxxxxxxxxxxxx>
> ---
> block/blk-core.c | 73 +++++++++++++++++++++++++++++++++++++--
> include/linux/blk_types.h | 66 +++++++++++++++++++++++++++++++++--
> include/linux/types.h | 2 +-
> 3 files changed, 135 insertions(+), 6 deletions(-)
>
> diff --git a/block/blk-core.c b/block/blk-core.c
> index 6d7d53030d7c..e5cd4ff08f5c 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -947,14 +947,22 @@ static blk_qc_t __submit_bio_noacct(struct bio *bio)
> {
> struct bio_list bio_list_on_stack[2];
> blk_qc_t ret = BLK_QC_T_NONE;
> + struct request_queue *top_q;
> + bool poll_on;
>
> BUG_ON(bio->bi_next);
>
> bio_list_init(&bio_list_on_stack[0]);
> current->bio_list = bio_list_on_stack;
>
> + top_q = bio->bi_bdev->bd_disk->queue;
> + poll_on = test_bit(QUEUE_FLAG_POLL, &top_q->queue_flags) &&
> + (bio->bi_opf & REQ_HIPRI);
> +
> do {
> - struct request_queue *q = bio->bi_bdev->bd_disk->queue;
> + blk_qc_t cookie;
> + struct block_device *bdev = bio->bi_bdev;
> + struct request_queue *q = bdev->bd_disk->queue;
> struct bio_list lower, same;
>
> if (unlikely(bio_queue_enter(bio) != 0))
> @@ -966,7 +974,23 @@ static blk_qc_t __submit_bio_noacct(struct bio *bio)
> bio_list_on_stack[1] = bio_list_on_stack[0];
> bio_list_init(&bio_list_on_stack[0]);
>
> - ret = __submit_bio(bio);
> + cookie = __submit_bio(bio);
> +
> + if (poll_on && blk_qc_t_valid(cookie)) {
In patch 8, dm_submit_bio() is changed to return BLK_QC_T_NONE always,
so the returned cookie may be BLK_QC_T_NONE for DM device, such as, in
case of DM_MAPIO_SUBMITTED returned from ->map(), and underlying bios
can be submitted from another context, then nothing is fed to blk_poll().
thanks,
Ming
