Hi Neil- > On Jun 11, 2019, at 7:42 PM, NeilBrown <neilb@xxxxxxxx> wrote: > > On Tue, Jun 11 2019, Chuck Lever wrote: > >> >> Earlier in this thread, Neil proposed to make nconnect a hint. Sounds >> like the long term plan is to allow "up to N" connections with some >> mechanism to create new connections on-demand." maxconn fits that idea >> better, though I'd prefer no new mount options... the point being that >> eventually, this setting is likely to be an upper bound rather than a >> fixed value. > > When I suggested making at I hint, I considered and rejected the the > idea of making it a maximum. Maybe I should have been explicit about > that. > > I think it *is* important to be able to disable multiple connections, > hence my suggestion that "nconnect=1", as a special case, could be a > firm maximum. > My intent was that if nconnect was not specified, or was given a larger > number, then the implementation should be free to use however many > connections it chose from time to time. The number given would be just > a hint - maybe an initial value. Neither a maximum nor a minimum. > Maybe we should add "nonconnect" (or similar) to enforce a single > connection, rather than overloading "nconnect=1" So then I think, for the immediate future, you want to see nconnect= specify the exact number of connections that will be opened. (later it can be something the client chooses automatically). IIRC that's what Trond's patches already do. Actually I prefer that the default behavior be the current behavior, where the client uses one connection per client-server pair. That serves the majority of use cases well enough. Saying that default is nconnect=1 is then intuitive to understand. At some later point if we convince ourselves that a higher default is safe (ie, does not result in performance regressions in some cases) then raise the default to nconnect=2 or 3. I'm not anxious to allow everyone to open an unlimited number of connections just yet. That has all kinds of consequences for servers, privileged port consumption, etc, etc. I'm not wont to hand an unlimited capability to admins who are not NFS-savvy in the name of experimentation. That will just make for more phone calls to our support centers and possibly some annoyed storage administrators. And it seems like something that can be abused pretty easily by certain ne'er-do-wells. Starting with a maximum of 3 or 4 is conservative yet exposes immediate benefits. The default connection behavior remains the same. No surprises when a stock Linux NFS client is upgraded to a kernel that supports nconnect. The maximum setting can be raised once we understand the corner cases, the benefits, and the pitfalls. > You have said elsewhere that you would prefer configuration in a config > file rather than as a mount option. > How do you imagine that configuration information getting into the > kernel? I'm assuming Trond's design, where the kernel RPC client upcalls to a user space agent (a new daemon, or request-key). > Do we create /sys/fs/nfs/something? or add to /proc/sys/sunrpc > or /proc/net/rpc .... we have so many options !! > There is even /sys/kernel/debug/sunrpc/rpc_clnt, but that is not > a good place for configuration. > > I suspect that you don't really have an opinion, you just don't like the > mount option. However I don't have that luxury. I need to put the > configuration somewhere. As it is per-server configuration the only > existing place that works at all is a mount option. > While that might not be ideal, I do think it is most realistic. > Mount options can be deprecated, and carrying support for a deprecated > mount option is not expensive. It's not deprecation that worries me, it's having to change the mount option; and the fact that we already believe it will have to change makes it especially worrisome that we are picking the wrong horse at the start. NFS mount options will appear in automounter maps for a very long time. They will be copied to other OSes. Deprecation is more expensive than you might at first think. > The option still can be placed in a per-server part of > /etc/nfsmount.conf rather than /etc/fstab, if that is what a sysadmin > wants to do. I don't see that having a mount option /and/ a configuration file addresses Trond's concern about config pulverization. It makes it worse, in fact. But my fundamental problem is with a per-server setting specified as a per-mount option. Using a config file is just a possible way to address that problem. For a moment, let's turn the mount option idea on its head. Another alternative would be to make nconnect into a real per-mount setting instead of a per-server setting. So now each mount gets to choose the number of connections it is permitted to use. Suppose we have three concurrent mounts: mount -o nconnect=3 server1:/export /mnt/one mount server2:/export /mnt/two mount -o nconnect=2 server3:/export /mnt/three The client opens the maximum of the three nconnect values, which is 3. Then: Traffic to server2 may use only one of these connections. Traffic to server3 may use no more than two of those connections. Traffic to server1 may use all three of those connections. Does that make more sense than a per-server setting? Is it feasible to implement? -- Chuck Lever
