> On Sep 19, 2022, at 1:59 PM, Trond Myklebust <trondmy@xxxxxxxxxxxxxxx> wrote: > > On Mon, 2022-09-19 at 15:31 +0000, Chuck Lever III wrote: >> Hi- >> >> I rediscovered recently that NFSv4.0 with Kerberos does not work on >> multi-homed hosts. This is true even with sec=sys because the client >> attempts to establish a GSS context when there is a keytab present. >> >> Basically my test environment has to work for sec=sys and sec=krb* >> and for all NFS versions and minor versions. Thus I keep a keytab >> on it. >> >> Now, I have three network interfaces on my client: one RoCE, one >> IB, and one GbE. They are each on their own subnet and each has >> a unique hostname (that varies in the domain part). >> >> But mounting one of my IB or RoCE test servers with NFSv4.0 results >> in the infamous "NFSv4: Invalid callback credential" message. The >> client always uses the principal for GbE interface. >> >> This was working at one point, but seems to have devolved over time. >> >> >> Here are some of the problems I found: >> >> 1. The kernel always asks for service=* . >> >> If your system's keytab has only "nfs" service principals in it, >> that should be OK. If it has a "host" principal in it, that's >> going to be the first one that gssd picks up. >> >> NFSv4.0 callback does not work with a host@ acceptor -- it wants >> nfs@. >> >> There are two possible workarounds: >> >> a. Remove all but the nfs@ keys from your system's keytab. >> >> b. Modify the kernel to use "service=nfs" in the upcall. >> > > There's also > > c. Put the nfs service principal in its own keytab and use the '-k' > option to tell rpc.gssd where to find it. > > However note that 'host/<hostname@REALM>' is normally the expected > principal name for authenticating as a specific hostname. So I'd expect > clients to want to authenticate using that credential so that it is > matched to the hostname entry in /etc/exports on the server. > > The 'nfs/<hostname@REALM>' would normally be considered a NFS service > principal name, so should really be used by the NFSv4 server to > identify its service (see RFC5661 Section 2.2.1.1.1.3.) rather than > being used by the NFS client. Fair enough, we can leave the client's service name alone. > The same principal is also used by the NFSv4 server to identify itself > when acting as a client to the NFS callback service according to > RFC7530 section 3.3.3. > So what I'm saying is that for the standard NFS client, then '*' is > probably the right thing to use (with a slight preference for 'host/'), > but for the NFS server use case of connecting to the callback service, > it should specify the 'nfs/' prefix. It can do that right now by > setting the clnt->cl_principal. As far as I can tell, the current > behaviour in knfsd is to set it to the same prefix as the server > svc_cred, and to default to 'nfs/' if the server svc_cred doesn't have > such a prefix. The server uses the client-provided service name in this case. If the client authenticates as "host@" then the server will authenticate to the "host@" service on the backchannel. Maybe the only mismatch is that my server is using "host@xxxxxxxxxxxxxxxxxxxxx" on the backchannel, and it should be using "host@xxxxxxxxxxxxxxxxxx" instead? -- Chuck Lever
