On Tue, Feb 15, 2022 at 04:13:25PM -0600, Patrick Goetz wrote: > When I set > > RPCNFSDCOUNT=16 > > what I thought this did was spin up an nfsd thread master with 15 > threads and the thread master would pass out client requests to the > threads, but looking at /proc/$PID/status -> TGID clearly indicates > these are all entirely separate processes. (I wasn't sure if ps > showed threads as separate processes; apparently it doesn't.) They're all kernel tasks, which makes the distinction between "thread" and "process" a little vague. > So the question is how do different client requests get farmed out > to different nfsd daemons for service? Who's actually listening on > port 2049? There's no user process that calls "listen"; knfsd's normal rpc handling is all in-kernel. Incoming rpc's may be handed to any of those 16 tasks for processing. A single task just runs a loop where it receives an rpc, handles it, and sends a response back. > This was all prompted by some vendor trying to sell me an EC > (Erasure Coding) n+m system who commented "NFS isn't multi-threaded, > NFS can only communicate with one server, for a shared/mounted > filesystem, so it will always be limited to the speed of that NFS > Server. POSIX/Multi-threaded means the filesystem is parallel and > can be reading/writing to multiple nodes at once in a storage > cluster/setup. The opposite of NFS." That explanation is a little muddled. NFS clients and servers both typically have lots of parallelism. Whether it's sufficient for your purposes depends on exactly what you need. But, yes, they're mostly correct to say that, in the absence of pNFS, "NFS can only communicate with one server, for a shared/mounted filesystem, so it will always be limited to the speed of that NFS Server". > I think pNFS addresses this, but then how does one implement pNFS? So, right, pNFS can let you perform IO to multiple servers simultaneously, if that's what you need. The Linux NFS client has support for pNFS, but the kernel server doesn't, so you'd need to look elsewhere for a pNFS server. Whether any of this is useful to you depends on exactly what problem you're trying to solve. --b.
