On Mon, 2022-09-19 at 15:31 +0000, Chuck Lever III wrote: > Hi- > > I rediscovered recently that NFSv4.0 with Kerberos does not work on > multi-homed hosts. This is true even with sec=sys because the client > attempts to establish a GSS context when there is a keytab present. > > Basically my test environment has to work for sec=sys and sec=krb* > and for all NFS versions and minor versions. Thus I keep a keytab > on it. > > Now, I have three network interfaces on my client: one RoCE, one > IB, and one GbE. They are each on their own subnet and each has > a unique hostname (that varies in the domain part). > > But mounting one of my IB or RoCE test servers with NFSv4.0 results > in the infamous "NFSv4: Invalid callback credential" message. The > client always uses the principal for GbE interface. > > This was working at one point, but seems to have devolved over time. > > > Here are some of the problems I found: > > 1. The kernel always asks for service=* . > > If your system's keytab has only "nfs" service principals in it, > that should be OK. If it has a "host" principal in it, that's > going to be the first one that gssd picks up. > > NFSv4.0 callback does not work with a host@ acceptor -- it wants > nfs@. > > There are two possible workarounds: > > a. Remove all but the nfs@ keys from your system's keytab. > > b. Modify the kernel to use "service=nfs" in the upcall. > There's also c. Put the nfs service principal in its own keytab and use the '-k' option to tell rpc.gssd where to find it. However note that 'host/<hostname@REALM>' is normally the expected principal name for authenticating as a specific hostname. So I'd expect clients to want to authenticate using that credential so that it is matched to the hostname entry in /etc/exports on the server. The 'nfs/<hostname@REALM>' would normally be considered a NFS service principal name, so should really be used by the NFSv4 server to identify its service (see RFC5661 Section 2.2.1.1.1.3.) rather than being used by the NFS client. The same principal is also used by the NFSv4 server to identify itself when acting as a client to the NFS callback service according to RFC7530 section 3.3.3. So what I'm saying is that for the standard NFS client, then '*' is probably the right thing to use (with a slight preference for 'host/'), but for the NFS server use case of connecting to the callback service, it should specify the 'nfs/' prefix. It can do that right now by setting the clnt->cl_principal. As far as I can tell, the current behaviour in knfsd is to set it to the same prefix as the server svc_cred, and to default to 'nfs/' if the server svc_cred doesn't have such a prefix. -- Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx
