On Mon, 2024-02-05 at 18:44 +0100, Lorenzo Bianconi wrote: > > On Mon, 2024-02-05 at 17:46 +0100, Lorenzo Bianconi wrote: > > > > On Fri, 2024-02-02 at 18:08 +0100, Lorenzo Bianconi wrote: > > > > > > The existing rpc.nfsd program was designed during a different time, when > > > > > > we just didn't require that much control over how it behaved. It's > > > > > > klunky to work with. > > > > > > > > > > > > In a response to Chuck's recent RFC patch to add knob to disable > > > > > > READ_PLUS calls, I mentioned that it might be a good time to make a > > > > > > clean break from the past and start a new program for controlling nfsd. > > > > > > > > > > > > Here's what I'm thinking: > > > > > > > > > > > > Let's build a swiss-army-knife kind of interface like git or virsh: > > > > > > > > > > > > # nfsdctl stats <--- fetch the new stats that got merged > > > > > > # nfsdctl add_listener <--- add a new listen socket, by address or hostname > > > > > > # nfsdctl set v3 on <--- enable NFSv3 > > > > > > # nfsdctl set splice_read off <--- disable splice reads (per Chuck's recent patch) > > > > > > # nfsdctl set threads 128 <--- spin up the threads > > > > > > > > > > > > We could start with just the bare minimum for now (the stats interface), > > > > > > and then expand on it. Once we're at feature parity with rpc.nfsd, we'd > > > > > > want to have systemd preferentially use nfsdctl instead of rpc.nfsd to > > > > > > start and stop the server. systemd will also need to fall back to using > > > > > > rpc.nfsd if nfsdctl or the netlink program isn't present. > > > > > > > > > > > > Note that I think this program will have to be a compiled binary vs. a > > > > > > python script or the like, given that it'll be involved in system > > > > > > startup. > > > > > > > > > > > > It turns out that Lorenzo already has a C program that has a lot of the > > > > > > plumbing we'd need: > > > > > > > > > > > > https://github.com/LorenzoBianconi/nfsd-netlink > > > > > > > > > > This is something I developed just for testing the new interface but I agree we > > > > > could start from it. > > > > > > > > > > Regarding the kernel part I addressed the comments I received upstream on v6 and > > > > > pushed the code here [0]. > > > > > How do you guys prefer to proceed? Is the better to post v7 upstream and continue > > > > > the discussion in order to have something usable to develop the user-space part or > > > > > do you prefer to have something for the user-space first? > > > > > I do not have a strong opinion on it. > > > > > > > > > > Regards, > > > > > Lorenzo > > > > > > > > > > [0] https://github.com/LorenzoBianconi/nfsd-next/tree/nfsd-next-netlink-new-cmds-public-v7 > > > > > > > > > > > > > > > > > > My advice? > > > > > > > > Step back and spend some time working on the userland bits before > > > > posting another revision. Experience has shown that you never realize > > > > what sort of warts an interface like this has until you have to work > > > > with it. > > > > > > > > You may find that you want to tweak it some once you do, and it's much > > > > easier to do that before we merge anything. This will be part of the > > > > kernel ABI, so once it's in a shipping kernel, we're sort of stuck with > > > > it. > > > > > > > > Having a userland program ready to go will allow us to do things like > > > > set up the systemd service for this too, which is primarily how this new > > > > program will be called. > > > > > > I agree on it. In order to proceed I guess we should define a list of > > > requirements/expected behaviour on this new user-space tool used to > > > configure nfsd. I am not so familiar with the user-space requirements > > > for nfsd so I am just copying what you suggested, something like: > > > > > > $ nfsdctl stats <--- fetch the new stats that got merged > > > $ nfsdctl xprt add proto <udp|tcp> host <host> [port <port>] <--- add a new listen socket, by address or hostname > > > > Those look fine. > > > > All of the commands should display the current state too when run with > > no arguments. So running "nfsctl xprt" should dump out all of the > > listening sockets. Ditto for the ones below too. > > ack > I think we might need a "nfsdctl xprt del" too. I know we don't have that functionality now, but I think it's missing. Otherwise if you mistakenly set the wrong socket with the interface above, how do you fix it? Alternately, we could provide some way to reset the server state altogether: # nfsdctl reset ? > > > > > $ nfsdctl proto v3.0 v4.0 v4.1 <--- enable NFSv3 and v4.1 > > > > The above would also enable v4.0 too? > > > > For this we might want to use the +/- syntax, actually. Also, there were > > no minorversions before v4, so it probably better not to accept that for > > v3: > > > > $ nfsdctl proto +v3 -v4.0 +4.1 > > according to the previous discussion we agreed to pass to the kernel just the > protocol versions we want to enable (v3.0 v4.0 v4.1 in the previous example) > and disable all the other ones..but I am fine to define a different semantics. > What do you think it is the best one? > The kernel interface should be declarative like you have, but the command-line interface could be more imperative like this. It could do a fetch of the currently enabled versions, twiddle them based on the +/- arguments and then send the new set down to the kernel for it to act on. One thing we haven't considered here too: Kconfig allows you to compile out support for some versions. For instance, v2 support is disabled by default these days. Maybe we should rethink the kernel interface too: When querying the versions, we could have it send the full set of all of the versions that it supports up to userland, and report whether each version is enabled or disabled. So if you get a list of versions from the kernel, and v2 isn't in the list at all, you can assume that the kernel doesn't support it. > > > > So to me, that would mean enable v3 and v4.1, and disable v4.0. > > v2 (if supported) and v4.2 would be left unchanged. > > > > > $ nfsdctl set threads 128 <--- spin up the threads > > > > > > > Maybe: > > > > $ nfsdctl threads 128 > > ack > > > > > ? > > > > > > > > > > > > > > > > > > > Can we start from [0] to develop them? > > > > > > > Yeah, that seems like a good place to start. > > ack, I will work in it. > Very exciting! -- Jeff Layton <jlayton@xxxxxxxxxx>
