On Tue, 2020-09-15 at 13:21 -0400, J. Bruce Fields wrote: > On Mon, Sep 07, 2020 at 06:31:00PM +0100, Daire Byrne wrote: > > 1) The kernel can drop entries out of the NFS client inode cache > > (under memory cache churn) when those filehandles are still being > > used by the knfsd's remote clients resulting in sporadic and random > > stale filehandles. This seems to be mostly for directories from > > what I've seen. Does the NFS client not know that knfsd is still > > using those files/dirs? The workaround is to never drop inode & > > dentry caches on the re-export servers (vfs_cache_pressure=1). This > > also helps to ensure that we actually make the most of our > > actimeo=3600,nocto mount options for the full specified time. > > I thought reexport worked by embedding the original server's > filehandles > in the filehandles given out by the reexporting server. > > So, even if nothing's cached, when the reexporting server gets a > filehandle, it should be able to extract the original filehandle from > it > and use that. > > I wonder why that's not working? NFSv3? If so, I suspect it is because we never wrote a lookupp() callback for it. > > > 4) With an NFSv4 re-export, lots of open/close requests (hundreds > > per > > second) quickly eat up the CPU on the re-export server and perf top > > shows we are mostly in native_queued_spin_lock_slowpath. > > Any statistics on who's calling that function? > > > Does NFSv4 > > also need an open file cache like that added to NFSv3? Our > > workaround > > is to either fix the thing doing lots of repeated open/closes or > > use > > NFSv3 instead. > > NFSv4 uses the same file cache. It might be the file cache that's at > fault, in fact.... > > --b. -- Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx
