Tried setting KRB5_CCNAME to KEYRING:persistent:1000 with no change
(Also tried KRB5CCNAME as well, as that's the environment variable
which seems to work with kinit & klist)
Also tried setting default_ccahe_name in /etc/krb5.conf to a legacy
value of
default_ccache_name = /tmp/krb5_%{uid}
in order to force it to use the filesystem rather than the kernel space
for the credentials cache. And it still fails in the same way. I think
that tends to makes this more likely a regression in krb5 than the kernel.
I believe that I may have verified this by reverting from
krb5-*/-/1.18.2-10 to krb5-*-1.17.1-5 which resolves the issue
On 07/21/2020 3:59 PM, Steve French wrote:
Would setting KRB5_CCNAME (as would be expected in the past) for that
process fix the issue?
On Tue, Jul 21, 2020 at 2:52 PM Michael Keane, K1MK <mkeane@xxxxxxxx> wrote:
KRB5_CCNAME is not set in the environment of the user for whom the
automount is failing.
In /etc/krb5.conf the default is set to:
default_ccache_name = KEYRING:persistent:%{uid}
The calls to keyctl() in the strace for cifs.upcall indicate that it's
successfully locating the user's keyring in the kernel and that it is
able to read key data from that keying
I haven't gone through and tried unpacking and decoding the contents of
those reads, but the reads aren't failing outright like it couldn't find
the credentials cache or the contents that it's looking for
On 07/21/2020 1:36 PM, Steve French wrote:
That is plausible but I also wonder about other whether other parts of
krb5 configuration are changed/broken eg KRB5_CCNAME environment variable
See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
On Tue, Jul 21, 2020, 12:24 Michael Keane, K1MK <mkeane@xxxxxxxx
<mailto:mkeane@xxxxxxxx>> wrote:
Thanks
In trying to further debug this problem, I've stood up a separate
Fedora 31 instance for testing with the result that the same
configuration that is failing on Fedora 32 is working under Fedora 31
Using strace it would appear the deviation occurs at the point of
"handle_krb5_mech" where the Fedora 31 instance proceeds though
several keyctl() calls to a successful return while the Fedora 32
instance goes through a similar sequence of keyctl() calls but
rather than finishing up with a final call to keyctl() to
instantiate the key it appears that the krb5 library and/or sssd
is going through a series of steps to locate the KDC (and failing)
even though the KDCs have always been explicitly configured in
/etc/krb5.conf
So the root cause of this issue may not be anything in cifs.upcall
but rather something in the sssd or kerberos that got changed as a
result of moving from Fedora 31 to Fedora 32
On 07/18/2020 8:40 PM, Steve French wrote:
Looks like error obtaining the key (either keyutils package not
installed or no Kerberos ticket found).
This page has debug instructions for cifs.upcall. see of that
information is helpful.
http://sprabhu.blogspot.com/2014/12/debugging-calls-to-cifsupcall.html?m=1
Also try kinit as a local user and then mount with cruid mount
option pointing to their uid to see if that helps.
On Fri, Jul 17, 2020, 09:04 Michael Keane, K1MK <mkeane@xxxxxxxx
<mailto:mkeane@xxxxxxxx>> wrote:
CIFS automounts to local NAS devices have stopped working
recently
Fedora 32
kernel 5.7.8-200.fc32.x86_64
mount.cifs version: 6.9
auto.misc: it-share
-fstype=cifs,multiuser,cruid=${UID},rw,vers=default,sec=krb5
://filer5/IT_Share
dmesg:
[ 3428.883661] fs/cifs/cifsfs.c: Devname:
//filer5.arrlhq.org/IT_Share
<http://filer5.arrlhq.org/IT_Share> flags: 0
[ 3428.883702] fs/cifs/connect.c: Username: root
[ 3428.883706] fs/cifs/connect.c: file mode: 0755 dir
mode: 0755
[ 3428.883709] fs/cifs/connect.c: CIFS VFS: in
mount_get_conns as
Xid: 12 with uid: 0
[ 3428.883710] fs/cifs/connect.c: UNC:
\\filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
[ 3428.883721] fs/cifs/connect.c: Socket created
[ 3428.883723] fs/cifs/connect.c: sndbuf 16384 rcvbuf 131072
rcvtimeo 0x1b58
[ 3428.884126] fs/cifs/fscache.c:
cifs_fscache_get_client_cookie:
(0x000000006d2b3226/0x0000000090edec3a)
[ 3428.884130] fs/cifs/connect.c: CIFS VFS: in
cifs_get_smb_ses as
Xid: 13 with uid: 0
[ 3428.884131] fs/cifs/connect.c: Existing smb sess not found
[ 3428.884135] fs/cifs/smb2pdu.c: Negotiate protocol
[ 3428.884141] fs/cifs/connect.c: Demultiplex PID: 5778
[ 3428.884156] fs/cifs/transport.c: Sending smb: smb_len=252
[ 3429.010818] fs/cifs/connect.c: RFC1002 header 0x11c
[ 3429.010830] fs/cifs/smb2misc.c: SMB2 data length 96
offset 128
[ 3429.010832] fs/cifs/smb2misc.c: SMB2 len 224
[ 3429.010835] fs/cifs/smb2misc.c: length of negcontexts
60 pad 0
[ 3429.010871] fs/cifs/transport.c: cifs_sync_mid_result:
cmd=0
mid=0 state=4
[ 3429.010885] fs/cifs/misc.c: Null buffer passed to
cifs_small_buf_release
[ 3429.010891] fs/cifs/smb2pdu.c: mode 0x1
[ 3429.010893] fs/cifs/smb2pdu.c: negotiated smb3.1.1 dialect
[ 3429.010903] fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2
0x348 0xbb92
[ 3429.010907] fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2
0x348 0x1bb92
[ 3429.010910] fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3
0x6 0x1
[ 3429.010912] fs/cifs/smb2pdu.c: decoding 2 negotiate
contexts
[ 3429.010914] fs/cifs/smb2pdu.c: decode SMB3.11
encryption neg
context of len 4
[ 3429.010916] fs/cifs/smb2pdu.c: SMB311 cipher type:1
[ 3429.010921] fs/cifs/connect.c: Security Mode: 0x1
Capabilities:
0x300046 TimeAdjust: 0
[ 3429.010923] fs/cifs/smb2pdu.c: Session Setup
[ 3429.010926] fs/cifs/smb2pdu.c: sess setup type 5
[ 3429.010949] fs/cifs/cifs_spnego.c: key description =
ver=0x2;host=filer5.arrlhq.org
<http://filer5.arrlhq.org>;ip4=10.1.123.38;sec=krb5;uid=0x0;creduid=0x3e8;user=root;pid=0x1690
[ 3429.025053] CIFS VFS: \\filer5.arrlhq.org
<http://filer5.arrlhq.org> Send error in SessSetup
= -126
[ 3429.025056] fs/cifs/connect.c: CIFS VFS: leaving
cifs_get_smb_ses
(xid = 13) rc = -126
[ 3429.025059] fs/cifs/connect.c: build_unc_path_to_root:
full_path=\\filer5.arrlhq.org
<http://filer5.arrlhq.org>\IT_Share
[ 3429.025059] fs/cifs/connect.c: build_unc_path_to_root:
full_path=\\filer5.arrlhq.org
<http://filer5.arrlhq.org>\IT_Share
[ 3429.025060] fs/cifs/connect.c: build_unc_path_to_root:
full_path=\\filer5.arrlhq.org
<http://filer5.arrlhq.org>\IT_Share
[ 3429.025062] fs/cifs/dfs_cache.c: __dfs_cache_find:
search path:
\filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
[ 3429.025063] fs/cifs/dfs_cache.c: get_dfs_referral: get
an DFS
referral for \filer5.arrlhq.org
<http://filer5.arrlhq.org>\IT_Share
[ 3429.025065] fs/cifs/dfs_cache.c: dfs_cache_noreq_find:
path:
\filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
[ 3429.025071] fs/cifs/fscache.c:
cifs_fscache_release_client_cookie:
(0x000000006d2b3226/0x0000000090edec3a)
[ 3429.025076] fs/cifs/connect.c: CIFS VFS: leaving
mount_put_conns
(xid = 12) rc = 0
[ 3429.025077] CIFS VFS: cifs_mount failed w/return code = -2
journalctl:
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: key
description:
cifs.spnego;0;0;39010000;ver=0x2;host=filer5.arrlhq.org
<http://filer5.arrlhq.org>;ip4=10.1.123.38;sec=krb5;uid=0x0;creduid=0x3e8;user=root;pid=0x1690
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: ver=2
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
host=filer5.arrlhq.org <http://filer5.arrlhq.org>
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: ip=10.1.123.38
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: sec=1
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: uid=0
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: creduid=1000
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: user=root
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: pid=5776
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
get_cachename_from_process_env: pathname=/proc/5776/environ
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
get_existing_cc: default ccache is
KEYRING:persistent:1000:1000
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
handle_krb5_mech: getting service ticket for
filer5.arrlhq.org <http://filer5.arrlhq.org>
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
cifs_krb5_get_req: unable to get credentials for
filer5.arrlhq.org <http://filer5.arrlhq.org>
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
handle_krb5_mech: failed to obtain service ticket
(-1765328370)
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: Unable to
obtain service ticket
Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
<http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: Exit status
-1765328370
$ klist
Ticket cache: KEYRING:persistent:1000:1000
Default principal: mkeane@xxxxxxxxxx <mailto:mkeane@xxxxxxxxxx>
Valid starting Expires Service principal
07/17/2020 09:43:06 07/17/2020 19:43:06
krbtgt/ARRLHQ.ORG@xxxxxxxxxx <mailto:ARRLHQ.ORG@xxxxxxxxxx>
renew until 07/24/2020 09:42:57
Filer5 is a QNAP TS-870U-RP Version 4.3.6.1070 (2019/09/10)
NAS device
but having this issue with other NAS device on LAN
--
Michael Keane, K1MK
IT Manager
ARRL, The National Association for Amateur Radio™
225 Main Street, Newington, CT 06111-1494 USA
Telephone: (860) 594-0285
email: mkeane@xxxxxxxx <mailto:mkeane@xxxxxxxx>
--
Michael Keane, K1MK
IT Manager
ARRL, The National Association for Amateur Radio™
225 Main Street, Newington, CT 06111-1494 USA
Telephone: (860) 594-0285
email:mkeane@xxxxxxxx <mailto:mkeane@xxxxxxxx>
--
Michael Keane, K1MK
IT Manager
ARRL, The National Association for Amateur Radio™
225 Main Street, Newington, CT 06111-1494 USA
Telephone: (860) 594-0285
email: mkeane@xxxxxxxx
--
Michael Keane, K1MK
IT Manager
ARRL, The National Association for Amateur Radio™
225 Main Street, Newington, CT 06111-1494 USA
Telephone: (860) 594-0285
email: mkeane@xxxxxxxx
