> > Our issue also happens when the writes are done on the fileserver, > > though, and they occur even if you allow plenty of time for the > > writes to settle. I can run my test program in a mode where it > > explicitly waits for me to tell it to continue, do the appending > > to the file on the fileserver, 'sync' on the fileserver, wait five > > minutes, and the NFS client will still see those zero bytes when it > > tries to read the new data. > > That's happening because we're not optimising for the broken case, and > instead we assume that we can cache data for as long as the file is > open and unlocked as indeed the close-to-open cache consistency model > has always stated that we can do. If I'm understanding all of this right, is what the kernel does more or less like this: when a NFS client program closes a writeable file (descriptor), the kernel flushes any pending writes, does a GETATTR afterward, and declares all current cached pages fully valid 'as of' that GETATTR result. When the file is reopened (in any mode), the kernel GETATTRs the file again; if the GETATTR hasn't changed, the cached pages and their contents remain valid. As a result, if you write to the file from another machine (including the fileserver) before the writeable file is closed, on close the client uses the updated GETATTR from the server but its current cached pages. These cached pages may be out of date, but if so it is because one violated close-to-open; you must always close any writeable file descriptors on machine A before writing to the file on machine B (or obtain and then release locks?). If a client kernel has cached pages this way, is there any simple sequence of system calls on the client that will cause it to discard these cached pages? Or do you need the file's GETATTR to change again, implicitly from another machine? (I assume that changing the file's attributes from the client with the cached pages doesn't cause it to invalidate them, and certainly eg a 'touch' doesn't do it from the client where it does do it from another machine.) - cks
