On Tue, 2021-08-03 at 21:51 -0400, Matt Benjamin wrote: > (who have performed an open) > > On Tue, Aug 3, 2021 at 9:43 PM Matt Benjamin <mbenjami@xxxxxxxxxx> > wrote: > > > > I think it is how close-to-open has been traditionally understood. > > I > > do not believe that close-to-open in any way implies a single > > writer, > > rather it sets the consistency expectation for all readers. > > OK. I'll bite, despite the obvious troll-bait... close-to-open implies a single writer because it is impossible to guarantee ordering semantics in RPC. You could, in theory, do so by serialising on the client, but none of us do that because we care about performance. If you don't serialise between clients, then it is trivial (and I'm seriously tired of people who whine about this) to reproduce reads to file areas that have not been fully synced to the server, despite having data on the client that is writing. i.e. the reader sees holes that never existed on the client that wrote the data. The reason is that the writes got re-ordered en route to the server, and so reads to the areas that have not yet been filled are showing up as holes. So, no, the close-to-open semantics definitely apply to both readers and writers. > > Matt > > > > On Tue, Aug 3, 2021 at 5:36 PM bfields@xxxxxxxxxxxx > > <bfields@xxxxxxxxxxxx> wrote: > > > > > > On Tue, Aug 03, 2021 at 09:07:11PM +0000, Trond Myklebust wrote: > > > > On Tue, 2021-08-03 at 16:30 -0400, J. Bruce Fields wrote: > > > > > On Fri, Jul 30, 2021 at 02:48:41PM +0000, Trond Myklebust > > > > > wrote: > > > > > > On Fri, 2021-07-30 at 09:25 -0400, Benjamin Coddington > > > > > > wrote: > > > > > > > I have some folks unhappy about behavior changes after: > > > > > > > 479219218fbe > > > > > > > NFS: > > > > > > > Optimise away the close-to-open GETATTR when we have > > > > > > > NFSv4 OPEN > > > > > > > > > > > > > > Before this change, a client holding a RO open would > > > > > > > invalidate > > > > > > > the > > > > > > > pagecache when doing a second RW open. > > > > > > > > > > > > > > Now the client doesn't invalidate the pagecache, though > > > > > > > technically > > > > > > > it could > > > > > > > because we see a changeattr update on the RW OPEN > > > > > > > response. > > > > > > > > > > > > > > I feel this is a grey area in CTO if we're already > > > > > > > holding an > > > > > > > open. > > > > > > > Do we > > > > > > > know how the client ought to behave in this case? Should > > > > > > > the > > > > > > > client's open > > > > > > > upgrade to RW invalidate the pagecache? > > > > > > > > > > > > > > > > > > > It's not a "grey area in close-to-open" at all. It is very > > > > > > cut and > > > > > > dried. > > > > > > > > > > > > If you need to invalidate your page cache while the file is > > > > > > open, > > > > > > then > > > > > > by definition you are in a situation where there is a write > > > > > > by > > > > > > another > > > > > > client going on while you are reading. You're clearly not > > > > > > doing > > > > > > close- > > > > > > to-open. > > > > > > > > > > Documentation is really unclear about this case. Every > > > > > definition of > > > > > close-to-open that I've seen says that it requires a cache > > > > > consistency > > > > > check on every application open. I've never seen one that > > > > > says "on > > > > > every open that doesn't overlap with an already-existing open > > > > > on that > > > > > client". > > > > > > > > > > They *usually* also preface that by saying that this is > > > > > motivated by > > > > > the > > > > > use case where opens don't overlap. But it's never made > > > > > clear that > > > > > that's part of the definition. > > > > > > > > > > > > > I'm not following your logic. > > > > > > It's just a question of what every source I can find says close- > > > to-open > > > means. E.g., NFS Illustrated, p. 248, "Close-to-open consistency > > > provides a guarantee of cache consistency at the level of file > > > opens and > > > closes. When a file is closed by an application, the client > > > flushes any > > > cached changs to the server. When a file is opened, the client > > > ignores > > > any cache time remaining (if the file data are cached) and makes > > > an > > > explicit GETATTR call to the server to check the file > > > modification > > > time." > > > > > > > The close-to-open model assumes that the file is only being > > > > modified by > > > > one client at a time and it assumes that file contents may be > > > > cached > > > > while an application is holding it open. > > > > The point checks exist in order to detect if the file is being > > > > changed > > > > when the file is not open. > > > > > > > > Linux does not have a per-application cache. It has a page > > > > cache that > > > > is shared among all applications. It is impossible for two > > > > applications > > > > to open the same file using buffered I/O, and yet see different > > > > contents. > > > > > > Right, so based on the descriptions like the one above, I would > > > have > > > expected both applications to see new data at that point. > > > > > > Maybe that's not practical to implement. It'd be nice at least > > > if that > > > was explicit in the documentation. > > > > > > --b. > > > > > > > > > -- > > > > Matt Benjamin > > Red Hat, Inc. > > 315 West Huron Street, Suite 140A > > Ann Arbor, Michigan 48103 > > > > http://www.redhat.com/en/technologies/storage > > > > tel. 734-821-5101 > > fax. 734-769-8938 > > cel. 734-216-5309 > > > -- Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx
