> On Feb 24, 2022, at 6:07 AM, Jeff Layton <jlayton@xxxxxxxxxx> wrote: > > On Thu, 2022-02-24 at 12:13 +0200, Amir Goldstein wrote: >> Hi Jeff, >> >> I got some reports from customers about failure to allocate the >> nfsd_file_hashtbl on nfs server restart on a long running system, >> probably due to memory fragmentation. >> >> A search in Google for this error message yields several results of >> similar reports [1][2]. >> >> My question is, does nfsd_file_cache_init() have to be done on server >> startup? >> >> Doesn't it make more sense to allocate all the memory pools and >> hash table on init_nfsd()? >> >> Thanks, >> Amir. >> >> [1] https://unix.stackexchange.com/questions/640534/nfs-cannot-allocate-memory >> [2] https://askubuntu.com/questions/1365821/nfs-crashing-on-ubuntu-server-20-04 > > That is a big allocation. On my box, nfsd_fcache_bucket is 80 bytes, so > we end up needing 80 contiguous pages to allocate the whole table. It > doesn't surprise me that it fails sometimes. > > We could just allocate it on init_nfsd, but that happens when the module > is plugged in and we'll lose 80 pages when people plug it in (or build > it in) and don't actually use nfsd. Reducing the bucket count might also help, especially if nfb_head were to be replaced with an rb_tree to allow more items in each bucket. > The other option might be to just use kvcalloc? It's not a frequent > allocation, so I don't think performance would be an issue. We had > similar reports several years ago with nfsd_reply_cache_init, and using > kvzalloc ended up taking care of it. A better long-term solution would be to not require a large allocation at all. Maybe we could consider some alternative data structures. -- Chuck Lever
