On Wed, Jan 27, 2021 at 09:44:50AM +0200, Maxim Mikityanskiy wrote:
> On Wed, Jan 27, 2021 at 6:23 AM Bart Van Assche <bvanassche@xxxxxxx> wrote:
> >
> > On 1/26/21 11:59 AM, Maxim Mikityanskiy wrote:
> > > The cited commit introduced a serious regression with SATA write speed,
> > > as found by bisecting. This patch reverts this commit, which restores
> > > write speed back to the values observed before this commit.
> > >
> > > The performance tests were done on a Helios4 NAS (2nd batch) with 4 HDDs
> > > (WD8003FFBX) using dd (bs=1M count=2000). "Direct" is a test with a
> > > single HDD, the rest are different RAID levels built over the first
> > > partitions of 4 HDDs. Test results are in MB/s, R is read, W is write.
> > >
> > > | Direct | RAID0 | RAID10 f2 | RAID10 n2 | RAID6
> > > ----------------+--------+-------+-----------+-----------+--------
> > > 9011495c9466 | R:256 | R:313 | R:276 | R:313 | R:323
> > > (before faulty) | W:254 | W:253 | W:195 | W:204 | W:117
> > > ----------------+--------+-------+-----------+-----------+--------
> > > 5ff9f19231a0 | R:257 | R:398 | R:312 | R:344 | R:391
> > > (faulty commit) | W:154 | W:122 | W:67.7 | W:66.6 | W:67.2
> > > ----------------+--------+-------+-----------+-----------+--------
> > > 5.10.10 | R:256 | R:401 | R:312 | R:356 | R:375
> > > unpatched | W:149 | W:123 | W:64 | W:64.1 | W:61.5
> > > ----------------+--------+-------+-----------+-----------+--------
> > > 5.10.10 | R:255 | R:396 | R:312 | R:340 | R:393
> > > patched | W:247 | W:274 | W:220 | W:225 | W:121
> > >
> > > Applying this patch doesn't hurt read performance, while improves the
> > > write speed by 1.5x - 3.5x (more impact on RAID tests). The write speed
> > > is restored back to the state before the faulty commit, and even a bit
> > > higher in RAID tests (which aren't HDD-bound on this device) - that is
> > > likely related to other optimizations done between the faulty commit and
> > > 5.10.10 which also improved the read speed.
> > >
> > > Signed-off-by: Maxim Mikityanskiy <maxtram95@xxxxxxxxx>
> > > Fixes: 5ff9f19231a0 ("block: simplify set_init_blocksize")
> > > Cc: Christoph Hellwig <hch@xxxxxx>
> > > Cc: Jens Axboe <axboe@xxxxxxxxx>
> > > ---
> > > fs/block_dev.c | 10 +++++++++-
> > > 1 file changed, 9 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/fs/block_dev.c b/fs/block_dev.c
> > > index 3b8963e228a1..235b5042672e 100644
> > > --- a/fs/block_dev.c
> > > +++ b/fs/block_dev.c
> > > @@ -130,7 +130,15 @@ EXPORT_SYMBOL(truncate_bdev_range);
> > >
> > > static void set_init_blocksize(struct block_device *bdev)
> > > {
> > > - bdev->bd_inode->i_blkbits = blksize_bits(bdev_logical_block_size(bdev));
> > > + unsigned int bsize = bdev_logical_block_size(bdev);
> > > + loff_t size = i_size_read(bdev->bd_inode);
> > > +
> > > + while (bsize < PAGE_SIZE) {
> > > + if (size & bsize)
> > > + break;
> > > + bsize <<= 1;
> > > + }
> > > + bdev->bd_inode->i_blkbits = blksize_bits(bsize);
> > > }
> > >
> > > int set_blocksize(struct block_device *bdev, int size)
> >
> > How can this patch affect write speed? I haven't found any calls of
> > set_init_blocksize() in the I/O path. Did I perhaps overlook something?
>
> I don't know the exact mechanism how this change affects the speed,
> I'm not an expert in the block device subsystem (I'm a networking
> guy). This commit was found by git bisect, and my performance test
> confirmed that reverting it fixes the bug.
>
> It looks to me as this function sets the block size as part of control
> flow, and this size is used later in the fast path, and the commit
> that removed the loop decreased this block size.
Right, the issue is stupid __block_write_full_page() which submits single bio
for each buffer head. And I have tried to improve the situation by merging
BHs into single bio, see below patch:
https://lore.kernel.org/linux-block/20201230000815.3448707-1-ming.lei@xxxxxxxxxx/
The above patch should improve perf for your test case.
--
Ming
