On Wed, 03 Jul 2024, Trond Myklebust wrote: > On Tue, 2024-07-02 at 23:04 +1000, Dave Chinner wrote: > > On Tue, Jul 02, 2024 at 12:33:53PM +0000, Trond Myklebust wrote: > > > On Mon, 2024-07-01 at 10:13 -0400, Mike Snitzer wrote: > > > > On Mon, Jul 01, 2024 at 09:46:36AM +1000, Dave Chinner wrote: > > > > > IMO, the only sane way to ensure this sort of nested "back-end > > > > > page > > > > > cleaning submits front-end IO filesystem IO" mechanism works is > > > > > to > > > > > do something similar to the loop device. You most definitely > > > > > don't > > > > > want to be doing buffered IO (double caching is almost always > > > > > bad) > > > > > and you want to be doing async direct IO so that the submission > > > > > thread is not waiting on completion before the next IO is > > > > > submitted. > > > > > > > > Yes, follow-on work is for me to revive the directio path for > > > > localio > > > > that ultimately wasn't pursued (or properly wired up) because it > > > > creates DIO alignment requirements on NFS client IO: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/commit/?h=nfs-localio-for-6.11-testing&id=f6c9f51fca819a8af595a4eb94811c1f90051eab > > > > I don't follow - this is page cache writeback. All the write IO from > > the bdi flusher thread should be page aligned, right? So why does DIO > > alignment matter here? > > > > There is no guarantee in NFS that writes from the flusher thread are > page aligned. If a page/folio is known to be up to date, we will > usually align writes to the boundaries, but we won't guarantee to do a > read-modify-write if that's not the case. Specifically, we will not do > so if the file is open for write-only. Would it be reasonable for the partial pages to be written over RPC and for only full pages to be sent directly to the server-side file using O_DIRECT writes? Presumably the benefits of localio are most pronounced with large writes which will mostly be full-page, or even full-folio. O_DIRECT writes on the NFS side would be more awkward. open(2) documents that NFS places no alignment restrictions on O_DIRECT I/O. If applications depend on that then some copying will have to be done before the data is written to a block filesystem - possibly into the page cache, possibly into some other buffer. This wouldn't be more copying that we already do. NeilBrown
