On Tue, Dec 28, 2021 at 04:11:51PM +1100, NeilBrown wrote: > This is due to an (arguable) weakness in the NFSv4 protocol. > In NFSv3 the reply to the WRITE request had "wcc" data which would > report change information before and after the write and, if present, it > was required to be atomic. So, providing timestamps had a high > resolution, the client0 would see change information from *before* the > write from client1 completed. So it would know it needed to refresh > after that write became visible. > > With NFSv4 there is no atomicity guarantees relating to writes and > changeid. > There is provision for atomicity around directory operations, but not > around data operations. > > This means that if different clients access a file concurrently, then > their cache can become incorrect. The only way to ensure uncorrupted > data is to use locking for ALL reads and writes. The above 'od -i' does > not perform a locked read, so can give incorrect data. > If you got a whole-file lock before reading, then you should get correct > data. You'd also have to get a whole-file write lock on every write, wouldn't you, to prevent your own write from obscuring the change-attribute update caused by a concurrent writer? > You could argue that this requirement (always lock if there is any risk) > is by design, and so this aspect of the protocl is not a weakness. The spec discussion of byte-range locking and caching is here: https://datatracker.ietf.org/doc/html/rfc7530#section-10.3.2 The nfs man page, under ac/noac, says "Using the noac option provides greater cache coherence among NFS clients accessing the same files, but it extracts a significant performance penalty. As such, judicious use of file locking is encouraged instead. The DATA AND METADATA COHERENCE section contains a detailed discussion of these trade-offs." That section does have a "Using file locks with NFS" subsection, but that subsection doesn't actually discuss the interaction of file locks with client caching. It's a confusing and under-documented area. --b.
