Thanks, Bruce. As always your responses are super informative. Just
one follow up question based strictly on curiosity:
On 2/16/22 13:22, J. Bruce Fields wrote:
On Tue, Feb 15, 2022 at 04:13:25PM -0600, Patrick Goetz wrote:
When I set
RPCNFSDCOUNT=16
what I thought this did was spin up an nfsd thread master with 15
threads and the thread master would pass out client requests to the
threads, but looking at /proc/$PID/status -> TGID clearly indicates
these are all entirely separate processes. (I wasn't sure if ps
showed threads as separate processes; apparently it doesn't.)
They're all kernel tasks, which makes the distinction between "thread"
and "process" a little vague.
So the question is how do different client requests get farmed out
to different nfsd daemons for service? Who's actually listening on
port 2049?
There's no user process that calls "listen"; knfsd's normal rpc handling
is all in-kernel. Incoming rpc's may be handed to any of those 16 tasks
for processing. A single task just runs a loop where it receives an
rpc, handles it, and sends a response back.
How does knfsd decide what user space nfsd process to hand a task off
to? Is it random, round robin, or something more sophisticated? Or does
it even matter if nfsd is only handling one request at a time anyway?
This was all prompted by some vendor trying to sell me an EC
(Erasure Coding) n+m system who commented "NFS isn't multi-threaded,
NFS can only communicate with one server, for a shared/mounted
filesystem, so it will always be limited to the speed of that NFS
Server. POSIX/Multi-threaded means the filesystem is parallel and
can be reading/writing to multiple nodes at once in a storage
cluster/setup. The opposite of NFS."
That explanation is a little muddled. NFS clients and servers both
typically have lots of parallelism. Whether it's sufficient for your
purposes depends on exactly what you need.
But, yes, they're mostly correct to say that, in the absence of pNFS,
"NFS can only communicate with one server, for a shared/mounted
filesystem, so it will always be limited to the speed of that NFS
Server".
I think pNFS addresses this, but then how does one implement pNFS?
So, right, pNFS can let you perform IO to multiple servers
simultaneously, if that's what you need.
The Linux NFS client has support for pNFS, but the kernel server
doesn't, so you'd need to look elsewhere for a pNFS server.
Whether any of this is useful to you depends on exactly what problem
you're trying to solve.
--b.
