Merging raw block device writes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



About 18 months ago, I raised an issue with merging of direct raw block
device writes. [1]   At the time, nobody offered any input on the issue.
To recap, when multiple direct write requests are in-flight to a raw block
device with I/O scheduler "none", and requests must wait for budget
(i.e., the device is SCSI), write requests don't go on a blk-mq software
queue, and no merging is done.  Direct read requests that must wait for
budget *do* go on a software queue and merges happen.

Recently, I noticed that the problem has been fixed in the latest
upstream kernel, and I had time to do further investigation on the
issue.  Bisecting shows the problem first occurred in 5.16-rc1 with
commit dc5fc361d891 from Jens Axboe.  This commit actually prevents
merging of both reads and writes.  But reads were indirectly fixed in
commit 54a88eb838d3 from Pavel Begunkov, also in 5.16-rc1, so
the read problem never occurred in a release.   There's no mention
of merging in either commit message, so I suspect the effect on
merging was unintentional in both cases.   In 5.16, blkdev_read_iter()
does not create a plug list, while blkdev_write_iter() does.  But the
lower level __blkdev_direct_IO() creates a plug list for both reads
and writes, which is why commit dc5fc361d891 broke both.  Then
commit 54a88eb838d3 bypassed __blkdev_direct_IO() in most
cases, and the new path does not create a plug list.  So reads
typically proceed without a plug list, and the merging can happen.
Writes still don't merge because of the plug list in the higher level
blkdev_write_iter().

The situation stayed that way until 6.5-rc1 when commit
712c7364655f from Christoph removed the plug list from
blkdev_write_iter().  Again, there's no mention of merging in the
commit message, so fixing the merge problem may be happenstance.

Hyper-V guests and the Azure cloud have a particular interest here
because Hyper-V guests uses SCSI as the standard interface to virtual
disks.  Azure cloud disks can be throttled to a limited number of IOPS,
so the number of in-flights I/Os can be relatively high, and
merging can be beneficial to staying within the throttle
limits.  Of the flip side, this problem hasn't generated complaints
over the last 18 months that I'm aware of, though that may be more
because commercial distros haven't been running 5.16 or later kernels
until relatively recently.

In any case, the 6.5 kernel fixes the problem, at least in the
common cases where there's no plug list.  But I still wonder if
there's a latent problem with the original commit dc5fc361d891
that should be looked at by someone with more blk-mq expertise
than I have.

Michael

[1] https://lore.kernel.org/linux-block/PH0PR21MB3025A7D1326A92A4B8BDB5FED7B59@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/





[Index of Archives]     [Linux RAID]     [Linux SCSI]     [Linux ATA RAID]     [IDE]     [Linux Wireless]     [Linux Kernel]     [ATH6KL]     [Linux Bluetooth]     [Linux Netdev]     [Kernel Newbies]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Device Mapper]

  Powered by Linux