> On May 10, 2018, at 1:40 PM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote: > > On Wed, May 9, 2018 at 5:19 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote: >> I'm right on the edge of my understanding of how this all works. >> >> I've re-keyed my NFS server. Now on my client, I'm seeing this on >> vers=4.0,sec=sys mounts: >> >> May 8 16:40:30 manet kernel: NFS: NFSv4 callback contains invalid cred >> May 8 16:40:30 manet kernel: NFS: NFSv4 callback contains invalid cred >> May 8 16:40:30 manet kernel: NFS: NFSv4 callback contains invalid cred >> >> manet is my client, and klimt is my server. I'm mounting with >> NFS/RDMA, so I'm mounting hostname klimt.ib, not klimt. >> >> Because the client is using krb5i for lease management, the server >> is required to use krb5i for the callback channel (S 3.3.3 of RFC >> 7530). >> >> After a SETCLIENTID, the client copies the acceptor from the GSS >> context it set up, and uses that to check incoming callback >> requests. I instrumented the client's SETCLIENTID proc, and I see >> this: >> >> check_gss_callback_principal: acceptor=nfs@xxxxxxxxxxxxxxxxxxxxxxxx, principal=host@xxxxxxxxxxxxxxxxxxxxx >> >> The principal strings are not equal, and that's why the client >> believes the callback credential is bogus. Now I'm trying to >> figure out whether it is the server's callback client or the >> client's callback server that is misbehaving. >> >> To me, the server's callback principal (host@klimt) seems like it >> is correct. The client would identify as host@manet when making >> calls to the server, for example, so I'd expect the server to >> behave similarly when performing callbacks. >> >> Can anyone shed more light on this? > > What are your full hostnames of each machine and does the reverse > lookup from the ip to hostname on each machine give you what you > expect? > > Sounds like all of them need to be resolved to <>.ib.1015grager.net > but somewhere you are getting <>.1015grager.net instead. The forward and reverse mappings are consistent, and rdns is disabled in my krb5.conf files. My server is multi-homed; it has a 1GbE interface (klimt.1015granger.net); an FDR IB interface (klimt.ib.1015granger.net); and a 25 GbE interface (klimt.roce.1015granger.net). My theory is that the server needs to use the same principal for callback operations that the client used for lease establishment. The last paragraph of S3.3.3 seems to state that requirement, though it's not especially clear; and the client has required it since commit f11b2a1cfbf5 (2014). So the server should authenticate as nfs@xxxxxxxx and not host@klimt, in this case, when performing callback requests. This seems to mean that the server stack is going to need to expose the SName in each GSS context so that it can dig that out to create a proper callback credential for each callback transport. I guess I've reported this issue before, but now I'm tucking in and trying to address it correctly. -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
