On Tue, Sep 18, 2018 at 10:42:03AM -0700, Stan Hu wrote: > Here are more traces that have been pared down by introducing a > 10-second sleep in the while loop: > > 1. Experiment 1: A normal rename from test2.txt to test.txt without a > file open: https://s3.amazonaws.com/gitlab-support/nfs/exp1-normal-rename-over-nfs.pcap > 2. Experiment 2: A rename from test2.txt to test.txt with a file open: > https://s3.amazonaws.com/gitlab-support/nfs/exp2-rename-over-nfs-with-file-open.pcap > 3. Experiment 3: Same as experiment 2 except an `ls` was issued in the > directory where the files reside: > https://s3.amazonaws.com/gitlab-support/nfs/exp3-rename-over-nfs-with-file-open-and-ls.pcap > > In the pcap of experiment 2, as before we see the NFS respond to the > first RENAME request with a NFS4ERR_DELAY. > > The next RENAME succeeds, and it looks to me that the GETATTR call for > the file handle receives a modification time of 3 minutes prior to the > current timestamp. In this case (rename within a directory, over an existing file), there are three objects affected: the file that's being renamed, the file that's renamed over, and the parent directory. Neither file should see it's mtime change: mtime only changes when a file's data changes. The ctime changes whenever datat or attributes change. I'm not sure whether the ctime of the renamed file is expected to change. The ctime of the renamed-over file will change (if for no other reason that that it's nlink value changes from 1 to 0). The directory mtime and ctime will both change, I think. In any case, what the client actually relies on here is that the rename changes the NFSv4 attribute called "Change". Depending on filesystem and kernel version that "Change" attribute may be derived from the ctime or from a separate counter. In any case, if that's not changing, there's a problem. I bet it is changing, though, so I doubt that's the bug here. > I could see how this might make sense since there > is an open file handle out there for this file. > Is this behavior up to the server implementation, and the Linux > implementation chooses to allow references to old handles? I see in > https://tools.ietf.org/html/rfc3010#page-153: > > A filehandle may or may not become stale or expire on a rename. > However, server implementors are strongly encouraged to attempt to > keep file handles from becoming stale or expiring in this fashion. > > In experiment 3, I see a burst of activity after the "ls" call. In > both experiments 1 and 3, I see more than 1 OPEN request for test.txt, > which seems to refresh the inode of test.txt. > > Any idea what's going on here? The filehandle of the renamed file should definitely stay good after the rename. The filehandle of the file that was renamed over (the old "test.txt") might or might not. I know the Linux server will still keep it around as long as an NFSv4 client has it opened and there's not a server reboot. Your isilon server might start returning STALE as soon as its unlinked by the rename, and that's probably an OK implementation choice. --b.
