On Fri, Oct 17, 2008 at 10:55:38AM -0400, William A. (Andy) Adamson wrote: > There is a related resource issue for the NFSv4.1 server DRC where the > server guarantees a per session DRC cache size. The server needs to > determine how much memory to assign to each session, and will > therefore need to limit the number of connections based not only on > the TCP buffer size, but also on memory. I'm writing the DRC code, and > I'm looking for suggestions on how to manage the per-session memory > resource. Any thought welcome.. As a starting point you could look at nfsd_create_serv() to see how it decides max_blksize. Caveat: Andrew Morton tells me we should be using nr_free_buffer_pages() here in place of totalram. Maybe we should define one function nfsd_how_big_should_i_be() that by default just returns some fraction of nr_free_buffer_pages(); and then use that return value to size various things like this. I don't know.... --b. > > Thanks > > -->Andy > > On Thu, Oct 16, 2008 at 8:14 PM, Neil Brown <neilb@xxxxxxx> wrote: > > On Thursday October 16, jlayton@xxxxxxxxxx wrote: > >> > >> Thanks for the info Neil, that helps clarify this... > >> > >> Using RLIMIT_NOFILE is an interesting idea. From a cursory look at the > >> code, the default for RLIMIT_NOFILE looks like it's generally 1024. > >> We'll have to figure that this limit will effectively act as a limit on > >> the number of concurrent lockd clients. It's not too hard to imagine a > >> server with more clients than this (think of a large compute cluster). > > > > If all those clients used UDP, this would not be a problem. While I > > see the value of TCP for NFS, it doesn't seem as convincing for NLM. > > > > But I don't expect we have the luxury of insisting that clients use > > UDP for locking :-( > > > >> > >> The problem as you mention, is that that limit won't be easily tunable. > >> I think we need some mechanism for an admin to tune this limit. It > >> doesn't have to be tunable on the fly, but shouldn't require a kernel > >> rebuild. We could eliminate this check for single-threaded services > >> entirely, but I suppose that leaves the door open for DoS attacks > >> against those services. > >> > >> Maybe the best thing is to go with Bruce's idea and add a sv_maxconn > >> field to the svc_serv struct. We could make that default to the max of > >> RLIMIT_NOFILE rlim_cur value or the currently calculated value. > >> Eventually we could add a mechanism to allow someone to tune that > >> value. A module parameter would probably be fine for lockd. We might > >> even want to set the limit lower for things like the nfsv4 callback > >> thread. > >> > >> Thoughts? > > > > A per-service setting that defaults to something reasonable like your > > suggestions and can be over-ridden by a module parameter sounds like a > > good idea. > > If you change the module parameter via > > /sys/modules/lockd/parameters/max_connections > > then it wouldn't take effect until the service were stopped and restarted, > > but I expect that is acceptable (and could probably be 'fixed' if really > > needed). > > > > NeilBrown > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > > the body of a message to majordomo@xxxxxxxxxxxxxxx > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
