Would setting KRB5_CCNAME (as would be expected in the past) for that
process fix the issue?
On Tue, Jul 21, 2020 at 2:52 PM Michael Keane, K1MK <mkeane@xxxxxxxx> wrote:
>
> KRB5_CCNAME is not set in the environment of the user for whom the
> automount is failing.
>
> In /etc/krb5.conf the default is set to:
>
> default_ccache_name = KEYRING:persistent:%{uid}
>
> The calls to keyctl() in the strace for cifs.upcall indicate that it's
> successfully locating the user's keyring in the kernel and that it is
> able to read key data from that keying
>
> I haven't gone through and tried unpacking and decoding the contents of
> those reads, but the reads aren't failing outright like it couldn't find
> the credentials cache or the contents that it's looking for
>
>
> On 07/21/2020 1:36 PM, Steve French wrote:
> > That is plausible but I also wonder about other whether other parts of
> > krb5 configuration are changed/broken eg KRB5_CCNAME environment variable
> >
> > See https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html
> >
> > On Tue, Jul 21, 2020, 12:24 Michael Keane, K1MK <mkeane@xxxxxxxx
> > <mailto:mkeane@xxxxxxxx>> wrote:
> >
> > Thanks
> >
> > In trying to further debug this problem, I've stood up a separate
> > Fedora 31 instance for testing with the result that the same
> > configuration that is failing on Fedora 32 is working under Fedora 31
> >
> > Using strace it would appear the deviation occurs at the point of
> > "handle_krb5_mech" where the Fedora 31 instance proceeds though
> > several keyctl() calls to a successful return while the Fedora 32
> > instance goes through a similar sequence of keyctl() calls but
> > rather than finishing up with a final call to keyctl() to
> > instantiate the key it appears that the krb5 library and/or sssd
> > is going through a series of steps to locate the KDC (and failing)
> > even though the KDCs have always been explicitly configured in
> > /etc/krb5.conf
> >
> > So the root cause of this issue may not be anything in cifs.upcall
> > but rather something in the sssd or kerberos that got changed as a
> > result of moving from Fedora 31 to Fedora 32
> >
> > On 07/18/2020 8:40 PM, Steve French wrote:
> >> Looks like error obtaining the key (either keyutils package not
> >> installed or no Kerberos ticket found).
> >>
> >> This page has debug instructions for cifs.upcall. see of that
> >> information is helpful.
> >>
> >> http://sprabhu.blogspot.com/2014/12/debugging-calls-to-cifsupcall.html?m=1
> >>
> >> Also try kinit as a local user and then mount with cruid mount
> >> option pointing to their uid to see if that helps.
> >>
> >> On Fri, Jul 17, 2020, 09:04 Michael Keane, K1MK <mkeane@xxxxxxxx
> >> <mailto:mkeane@xxxxxxxx>> wrote:
> >>
> >> CIFS automounts to local NAS devices have stopped working
> >> recently
> >>
> >> Fedora 32
> >>
> >> kernel 5.7.8-200.fc32.x86_64
> >>
> >> mount.cifs version: 6.9
> >>
> >> auto.misc: it-share
> >> -fstype=cifs,multiuser,cruid=${UID},rw,vers=default,sec=krb5
> >> ://filer5/IT_Share
> >>
> >> dmesg:
> >>
> >> [ 3428.883661] fs/cifs/cifsfs.c: Devname:
> >> //filer5.arrlhq.org/IT_Share
> >> <http://filer5.arrlhq.org/IT_Share> flags: 0
> >> [ 3428.883702] fs/cifs/connect.c: Username: root
> >> [ 3428.883706] fs/cifs/connect.c: file mode: 0755 dir
> >> mode: 0755
> >> [ 3428.883709] fs/cifs/connect.c: CIFS VFS: in
> >> mount_get_conns as
> >> Xid: 12 with uid: 0
> >> [ 3428.883710] fs/cifs/connect.c: UNC:
> >> \\filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
> >> [ 3428.883721] fs/cifs/connect.c: Socket created
> >> [ 3428.883723] fs/cifs/connect.c: sndbuf 16384 rcvbuf 131072
> >> rcvtimeo 0x1b58
> >> [ 3428.884126] fs/cifs/fscache.c:
> >> cifs_fscache_get_client_cookie:
> >> (0x000000006d2b3226/0x0000000090edec3a)
> >> [ 3428.884130] fs/cifs/connect.c: CIFS VFS: in
> >> cifs_get_smb_ses as
> >> Xid: 13 with uid: 0
> >> [ 3428.884131] fs/cifs/connect.c: Existing smb sess not found
> >> [ 3428.884135] fs/cifs/smb2pdu.c: Negotiate protocol
> >> [ 3428.884141] fs/cifs/connect.c: Demultiplex PID: 5778
> >> [ 3428.884156] fs/cifs/transport.c: Sending smb: smb_len=252
> >> [ 3429.010818] fs/cifs/connect.c: RFC1002 header 0x11c
> >> [ 3429.010830] fs/cifs/smb2misc.c: SMB2 data length 96
> >> offset 128
> >> [ 3429.010832] fs/cifs/smb2misc.c: SMB2 len 224
> >> [ 3429.010835] fs/cifs/smb2misc.c: length of negcontexts
> >> 60 pad 0
> >> [ 3429.010871] fs/cifs/transport.c: cifs_sync_mid_result:
> >> cmd=0
> >> mid=0 state=4
> >> [ 3429.010885] fs/cifs/misc.c: Null buffer passed to
> >> cifs_small_buf_release
> >> [ 3429.010891] fs/cifs/smb2pdu.c: mode 0x1
> >> [ 3429.010893] fs/cifs/smb2pdu.c: negotiated smb3.1.1 dialect
> >> [ 3429.010903] fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2
> >> 0x348 0xbb92
> >> [ 3429.010907] fs/cifs/asn1.c: OID len = 7 oid = 0x1 0x2
> >> 0x348 0x1bb92
> >> [ 3429.010910] fs/cifs/asn1.c: OID len = 10 oid = 0x1 0x3
> >> 0x6 0x1
> >> [ 3429.010912] fs/cifs/smb2pdu.c: decoding 2 negotiate
> >> contexts
> >> [ 3429.010914] fs/cifs/smb2pdu.c: decode SMB3.11
> >> encryption neg
> >> context of len 4
> >> [ 3429.010916] fs/cifs/smb2pdu.c: SMB311 cipher type:1
> >> [ 3429.010921] fs/cifs/connect.c: Security Mode: 0x1
> >> Capabilities:
> >> 0x300046 TimeAdjust: 0
> >> [ 3429.010923] fs/cifs/smb2pdu.c: Session Setup
> >> [ 3429.010926] fs/cifs/smb2pdu.c: sess setup type 5
> >> [ 3429.010949] fs/cifs/cifs_spnego.c: key description =
> >> ver=0x2;host=filer5.arrlhq.org
> >> <http://filer5.arrlhq.org>;ip4=10.1.123.38;sec=krb5;uid=0x0;creduid=0x3e8;user=root;pid=0x1690
> >> [ 3429.025053] CIFS VFS: \\filer5.arrlhq.org
> >> <http://filer5.arrlhq.org> Send error in SessSetup
> >> = -126
> >> [ 3429.025056] fs/cifs/connect.c: CIFS VFS: leaving
> >> cifs_get_smb_ses
> >> (xid = 13) rc = -126
> >> [ 3429.025059] fs/cifs/connect.c: build_unc_path_to_root:
> >> full_path=\\filer5.arrlhq.org
> >> <http://filer5.arrlhq.org>\IT_Share
> >> [ 3429.025059] fs/cifs/connect.c: build_unc_path_to_root:
> >> full_path=\\filer5.arrlhq.org
> >> <http://filer5.arrlhq.org>\IT_Share
> >> [ 3429.025060] fs/cifs/connect.c: build_unc_path_to_root:
> >> full_path=\\filer5.arrlhq.org
> >> <http://filer5.arrlhq.org>\IT_Share
> >> [ 3429.025062] fs/cifs/dfs_cache.c: __dfs_cache_find:
> >> search path:
> >> \filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
> >> [ 3429.025063] fs/cifs/dfs_cache.c: get_dfs_referral: get
> >> an DFS
> >> referral for \filer5.arrlhq.org
> >> <http://filer5.arrlhq.org>\IT_Share
> >> [ 3429.025065] fs/cifs/dfs_cache.c: dfs_cache_noreq_find:
> >> path:
> >> \filer5.arrlhq.org <http://filer5.arrlhq.org>\IT_Share
> >> [ 3429.025071] fs/cifs/fscache.c:
> >> cifs_fscache_release_client_cookie:
> >> (0x000000006d2b3226/0x0000000090edec3a)
> >> [ 3429.025076] fs/cifs/connect.c: CIFS VFS: leaving
> >> mount_put_conns
> >> (xid = 12) rc = 0
> >> [ 3429.025077] CIFS VFS: cifs_mount failed w/return code = -2
> >>
> >> journalctl:
> >>
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: key
> >> description:
> >> cifs.spnego;0;0;39010000;ver=0x2;host=filer5.arrlhq.org
> >> <http://filer5.arrlhq.org>;ip4=10.1.123.38;sec=krb5;uid=0x0;creduid=0x3e8;user=root;pid=0x1690
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: ver=2
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> >> host=filer5.arrlhq.org <http://filer5.arrlhq.org>
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: ip=10.1.123.38
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: sec=1
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: uid=0
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: creduid=1000
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: user=root
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: pid=5776
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> >> get_cachename_from_process_env: pathname=/proc/5776/environ
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> >> get_existing_cc: default ccache is
> >> KEYRING:persistent:1000:1000
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> >> handle_krb5_mech: getting service ticket for
> >> filer5.arrlhq.org <http://filer5.arrlhq.org>
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> >> cifs_krb5_get_req: unable to get credentials for
> >> filer5.arrlhq.org <http://filer5.arrlhq.org>
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]:
> >> handle_krb5_mech: failed to obtain service ticket
> >> (-1765328370)
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: Unable to
> >> obtain service ticket
> >> Jul 17 09:43:13 mkZ230.ARRLHQ.ORG
> >> <http://mkZ230.ARRLHQ.ORG> cifs.upcall[5779]: Exit status
> >> -1765328370
> >>
> >> $ klist
> >> Ticket cache: KEYRING:persistent:1000:1000
> >> Default principal: mkeane@xxxxxxxxxx <mailto:mkeane@xxxxxxxxxx>
> >>
> >> Valid starting Expires Service principal
> >> 07/17/2020 09:43:06 07/17/2020 19:43:06
> >> krbtgt/ARRLHQ.ORG@xxxxxxxxxx <mailto:ARRLHQ.ORG@xxxxxxxxxx>
> >> renew until 07/24/2020 09:42:57
> >>
> >> Filer5 is a QNAP TS-870U-RP Version 4.3.6.1070 (2019/09/10)
> >> NAS device
> >> but having this issue with other NAS device on LAN
> >>
> >> --
> >> Michael Keane, K1MK
> >> IT Manager
> >> ARRL, The National Association for Amateur Radio™
> >> 225 Main Street, Newington, CT 06111-1494 USA
> >> Telephone: (860) 594-0285
> >> email: mkeane@xxxxxxxx <mailto:mkeane@xxxxxxxx>
> >>
> >
> > --
> > Michael Keane, K1MK
> > IT Manager
> > ARRL, The National Association for Amateur Radio™
> > 225 Main Street, Newington, CT 06111-1494 USA
> > Telephone: (860) 594-0285
> > email:mkeane@xxxxxxxx <mailto:mkeane@xxxxxxxx>
> >
>
> --
> Michael Keane, K1MK
> IT Manager
> ARRL, The National Association for Amateur Radio™
> 225 Main Street, Newington, CT 06111-1494 USA
> Telephone: (860) 594-0285
> email: mkeane@xxxxxxxx
>
--
Thanks,
Steve
