RE: Use of READDIRPLUS on large directories

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Perhaps the use of a heuristic that enables readdirplus only after the application has shown that it is interested in the attributes for each entry in the directory?  Thus, if the application does readdir()/stat()/stat()/stat()/readdir()/... then the NFS client could use readdirplus to fill the caches.  If the application is just reading the directory and looking at the names, then the client could just use readdir.

		ps


-----Original Message-----
From: linux-nfs-owner@xxxxxxxxxxxxxxx [mailto:linux-nfs-owner@xxxxxxxxxxxxxxx] On Behalf Of NeilBrown
Sent: Wednesday, March 16, 2011 12:55 AM
To: Trond Myklebust; Bryan Schumaker
Cc: linux-nfs@xxxxxxxxxxxxxxx
Subject: Use of READDIRPLUS on large directories


Hi Trond / Bryan et al.

Now that openSUSE 11.4 is out I have started getting a few reports
of regressions that can be traced to 

commit 0715dc632a271fc0fedf3ef4779fe28ac1e53ef4
Author: Bryan Schumaker <bjschuma@xxxxxxxxxx>
Date:   Fri Sep 24 18:50:01 2010 -0400

    NFS: remove readdir plus limit
    
    We will now use readdir plus even on directories that are very large.
    
    Signed-off-by: Bryan Schumaker <bjschuma@xxxxxxxxxx>
    Signed-off-by: Trond Myklebust <Trond.Myklebust@xxxxxxxxxx>


This particularly affects users with their home directory over
NFS, and with largish maildir mail folders.

Where it used to take a smallish number of seconds for (e.g.)
xbiff to start up and read through the various directories, it now
takes multiple minutes.

I can confirm that the slow down is due to readdirplus by mounting the
filesystem with nordirplus.


While I can understand that there are sometime benefits in using
readdirplus for very large directories, there are also obviously real
costs.  So I think we have to see this patch as a regression that should
be reverted.


It would quite possibly make sense to create a tunable (mount option or
sysctl I guess) to set the max size for directories to use readdirplus,
but I think it really should be an opt-in situation.

[[ It would also be really nice if the change-log for such a significant
change contained a little more justification.... :-(   ]]

Thoughts?

Thanks,
NeilBrown
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux