On Wed, 2006-04-26 at 09:14 -0400, Peter Staubach wrote: > Trond Myklebust wrote: > > >On Tue, 2006-04-25 at 21:14 -0400, Steve Dickson wrote: > > > > > >>Currently the NFS client caches ACCESS information on a per uid basis > >>which fall apart when different process with different uid consistently > >>access the same directory. The end result being a storm of needless > >>ACCESS calls... > >> > >>The attached patch used a hash table to store the nfs_access_entry > >>entires which cause the ACCESS request to only happen when the > >>attributes timeout.. The table is indexed by the addition of the > >>nfs_inode pointer and the cr_uid in the cred structure which should > >>spread things out nicely for some decent scalability (although the > >>locking scheme may need to be reworked a bit). The table has 256 entries > >>of struct list_head giving it a total size of 2k. > >> > >> > > > >Instead of having the field 'id', why don't you let the nfs_inode keep a > >small (hashed?) list of all the nfs_access_entry objects that refer to > >it? That would speed up searches for cached entries. > > > >I agree with Neil's assessment that we need a bound on the size of the > >cache. In fact, enforcing a bound is pretty much the raison d'être for a > >global table (by which I mean that if we don't need a bound, then we > >might as well cache everything in the nfs_inode). > >How about rather changing that hash table into an LRU list, then adding > >a shrinker callback (using set_shrinker()) to allow the VM to free up > >entries when memory pressure dictates that it must? > > > > Previous implementations have shown that a single per inode linear > linked list > ends up not being scalable enough in certain situations. There would end up > being too many entries in the list and searching the list would become > a bottleneck. Adding a set of hash buckets per inode also proved to be > inefficient because in order to have enough hash buckets to make the hashing > efficient, much space was wasted. Having a single set of hash buckets, > adequately sized, ended up being the best solution. What situations? AFAIA the number of processes in a typical setup are almost always far smaller than the number of cached inodes. For instance on my laptop, I'm currently running 146 processes, but according to /proc/slabinfo I'm caching 330000 XFS inodes + 141500 ext3 inodes. If I were to assume that a typical nfsroot system will show roughly the same behaviour, then it would mean that a typical bucket in Steve's 256 hash entry table will contain at least 2000 entries that I need to search through every time I want to do an access call. > Why have a limit? A better solution is to have the cache grow dynamically > as need requires and then have the system reclaim the memory when it starts > to run low on memory. See my previous mail: that is exactly what I suggested in the 3rd paragraph. Cheers, Trond - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
