On 19/06/2024 17:15, Trond Myklebust wrote:
On Wed, 2024-06-19 at 11:37 +0300, Sagi Grimberg wrote:
On 18/06/2024 21:59, Trond Myklebust wrote:
Hi Dan,
On Tue, 2024-06-18 at 18:33 +0300, Dan Aloni wrote:
There are some applications that write to predefined non-
overlapping
file offsets from multiple clients and therefore don't need to
rely
on
file locking. However, NFS file system behavior of extending
writes
to
to deal with write fragmentation, causes those clients to corrupt
each
other's data.
To help these applications, this change adds the `noextend`
parameter
to
the mount options, and handles this case in
`nfs_can_extend_write`.
Clients can additionally add the 'noac' option to ensure page
cache
flush on read for modified files.
I'm not overly enamoured of the name "noextend". To me that sounds
like
it might have something to do with preventing appends. Can we find
something that is a bit more descriptive?
nopbw (No page boundary writes) ?
That said, and given your last comment about reads. Wouldn't it be
better to have the application use O_DIRECT for these workloads?
Turning off attribute caching is both racy and an inefficient way
to
manage page cache consistency. It forces the client to bombard the
server with GETATTR requests in order to check that the page cache
is
in synch, whereas your description of the workload appears to
suggest
that the correct assumption should be that it is not in synch.
IOW: I'm asking if the better solution might not be to rather
implement
something akin to Solaris' "forcedirectio"?
This access pattern represents a common case in HPC where different
workers
write records to a shared output file which do not necessarily align
to
a page boundary.
This is not everything that the app is doing nor the only file it is
accessing, so IMO forcing
directio universally is may penalize the application.
Worse than forcing an attribute revalidation on every read?
For this use-case, yes. Different workloads may or may not be interested
in reading
this file.
BTW: We've been asked about the same issue from some of our customers,
and are planning on solving the problem by adding a new per-file
attribute to the NFSv4.2 protocol.
Interesting, I recently joined the ietf mailing list but have not seen
discussion
this as of yet. Would be interested to learn more.
The detection of that NOCACHE attribute would cause the client to
automatically choose O_DIRECT on file open, overriding the default
buffered I/O model. So this would allow the user or sysadmin to specify
at file creation time that this file will be used for purposes that are
incompatible with caching.
user/sysadmin as in not the client? setting this out-of-band?
That does not work where the application and the sysadmin do not know about
each other (i.e. in a cloud environment).
The use-case that is described here cannot be mandated by the server because
the file usage pattern is really driven by the application.
