On Thu, May 10, 2018 at 2:09 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote:
>
>
>> On May 10, 2018, at 1:40 PM, Olga Kornievskaia <aglo@xxxxxxxxx> wrote:
>>
>> On Wed, May 9, 2018 at 5:19 PM, Chuck Lever <chuck.lever@xxxxxxxxxx> wrote:
>>> I'm right on the edge of my understanding of how this all works.
>>>
>>> I've re-keyed my NFS server. Now on my client, I'm seeing this on
>>> vers=4.0,sec=sys mounts:
>>>
>>> May 8 16:40:30 manet kernel: NFS: NFSv4 callback contains invalid cred
>>> May 8 16:40:30 manet kernel: NFS: NFSv4 callback contains invalid cred
>>> May 8 16:40:30 manet kernel: NFS: NFSv4 callback contains invalid cred
>>>
>>> manet is my client, and klimt is my server. I'm mounting with
>>> NFS/RDMA, so I'm mounting hostname klimt.ib, not klimt.
>>>
>>> Because the client is using krb5i for lease management, the server
>>> is required to use krb5i for the callback channel (S 3.3.3 of RFC
>>> 7530).
>>>
>>> After a SETCLIENTID, the client copies the acceptor from the GSS
>>> context it set up, and uses that to check incoming callback
>>> requests. I instrumented the client's SETCLIENTID proc, and I see
>>> this:
>>>
>>> check_gss_callback_principal: acceptor=nfs@xxxxxxxxxxxxxxxxxxxxxxxx, principal=host@xxxxxxxxxxxxxxxxxxxxx
>>>
>>> The principal strings are not equal, and that's why the client
>>> believes the callback credential is bogus. Now I'm trying to
>>> figure out whether it is the server's callback client or the
>>> client's callback server that is misbehaving.
>>>
>>> To me, the server's callback principal (host@klimt) seems like it
>>> is correct. The client would identify as host@manet when making
>>> calls to the server, for example, so I'd expect the server to
>>> behave similarly when performing callbacks.
>>>
>>> Can anyone shed more light on this?
>>
>> What are your full hostnames of each machine and does the reverse
>> lookup from the ip to hostname on each machine give you what you
>> expect?
>>
>> Sounds like all of them need to be resolved to <>.ib.1015grager.net
>> but somewhere you are getting <>.1015grager.net instead.
>
> The forward and reverse mappings are consistent, and rdns is
> disabled in my krb5.conf files. My server is multi-homed; it
> has a 1GbE interface (klimt.1015granger.net); an FDR IB
> interface (klimt.ib.1015granger.net); and a 25 GbE interface
> (klimt.roce.1015granger.net).
Ah, so you are keeping it very interesting...
> My theory is that the server needs to use the same principal
> for callback operations that the client used for lease
> establishment. The last paragraph of S3.3.3 seems to state
> that requirement, though it's not especially clear; and the
> client has required it since commit f11b2a1cfbf5 (2014).
>
> So the server should authenticate as nfs@xxxxxxxx and not
> host@klimt, in this case, when performing callback requests.
Yes I agree that server should have authenticated as nfs@xxxxxxxx and
that's what I see in my (simple) single home setup.
In nfs-utils there is code that deals with the callback and comment
about choices for the principal:
* Restricting gssd to use "nfs" service name is needed for when
* the NFS server is doing a callback to the NFS client. In this
* case, the NFS server has to authenticate itself as "nfs" --
* even if there are other service keys such as "host" or "root"
* in the keytab.
So the upcall for the callback should have specifically specified
"nfs" to look for the nfs/<hostname>. Question is if you key tab has
both:
nfs/klmit and nfs/klmit.ib how does it choose which one to take. I'm
not sure. But I guess in your case you are seeing that it choose
"host/<>" which would really be a nfs-utils bug.
What's in your server's key tab?
An output from gssd -vvv would be interesting.
> This seems to mean that the server stack is going to need to
> expose the SName in each GSS context so that it can dig that
> out to create a proper callback credential for each callback
> transport.
>
> I guess I've reported this issue before, but now I'm tucking
> in and trying to address it correctly.
>
>
> --
> Chuck Lever
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
