On Fri, May 19, 2023 at 10:55:29AM +0800, Wang Yugui wrote:
> Hi,
>
> > On Thu, May 18, 2023 at 10:46:43PM +0100, Matthew Wilcox wrote:
> > > -struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos)
> > > +struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
> > > {
> > > unsigned fgp = FGP_WRITEBEGIN | FGP_NOFS;
> > > + struct folio *folio;
> > >
> > > if (iter->flags & IOMAP_NOWAIT)
> > > fgp |= FGP_NOWAIT;
> > > + fgp |= fgp_order(len);
> > >
> > > - return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
> > > + folio = __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
> > > fgp, mapping_gfp_mask(iter->inode->i_mapping));
> > > + if (!IS_ERR(folio) && folio_test_large(folio))
> > > + printk("index:%lu len:%zu order:%u\n", (unsigned long)(pos / PAGE_SIZE), len, folio_order(folio));
> > > + return folio;
> > > }
> >
> > Forgot to take the debugging out. This should read:
> >
> > -struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos)
> > +struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
> > {
> > unsigned fgp = FGP_WRITEBEGIN | FGP_NOFS;
> > if (iter->flags & IOMAP_NOWAIT)
> > fgp |= FGP_NOWAIT;
> > + fgp |= fgp_order(len);
> >
> > return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
> > fgp, mapping_gfp_mask(iter->inode->i_mapping));
> > }
>
> I test it (attachment file) on 6.4.0-rc2.
> fio -name write-bandwidth -rw=write -bs=1024Ki -size=32Gi -runtime=30 -iodepth 1 -ioengine sync -zero_buffers=1 -direct=0 -end_fsync=1 -numjobs=4 -directory=/mnt/test
>
> fio WRITE: bw=2430MiB/s.
> expected value: > 6000MiB/s
> so yet no fio write bandwidth improvement.
That's basically unchanged. There's no per-page or per-block work being
done in start/end writeback, so if Dave's investigation is applicable
to your situation, I'd expect to see an improvement.
Maybe try the second version of the patch I sent with the debug in,
to confirm you really are seeing large folios being created (you might
want to use printk_ratelimit() instead of printk to ensure it doesn't
overwhelm your system)? That fio command you were using ought to create
them, but there's always a chance it doesn't.
Perhaps you could use perf (the command Dave used) to see where the time
is being spent.
