On Tue, 13 Jan 2015, Julien Cristau wrote: > Hi, > > we're experiencing some troubling performance issues with our nfs clients. One > of the symptoms is degraded performance of directory walks with hot caches and > expired attribute caches: "find -ls" in the kernel tree goes from about 4s > (cold caches) to 1.5s (hot vm caches) and then to 18s after actimeo. Network > traffic increases by about the same amount. "echo 3 > > /proc/sys/vm/drop_caches" brings back the initial performance (~4s), but so do > writing 1 or 2. > > This was reported almost two years ago in https://bugs.debian.org/702477 but I > don't know if this is a known issue upstream. Seen on various kernel versions, > up to 3.18. > > Is there anything we can do to track this down, or are there known workarounds > for this? > > Thanks, > Julien Hi Julien, I think that after the attribute cache times out, the client has to do a lot of GETATTRs to revalidate inodes which is where you're getting slowed down. When each directory is initially read READDIR is filling in the attribute information and then caching the results of that READDIR in the page cache. Then the attribute cache times out, the directory is revalidated (it hasn't changed) and the contents of the directory are read from page cache, but each directory entry's attributes are now also timed out and so the many GETATTRs. When you dump the page cache, it makes sense that the client goes back to doing READDIRs, and your directory walk is faster. Possibly we could invalidate directory inode pages whenever the attributes time out, but I'm not sure what all the ramifications of that may be. I also feel like this issue has already been discussed, but I'm not immediately able to locate that discussion. Ben -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
