Hello everyone,
We have a CentOS 6.3 NFSv4 server and client, and we've run into a situation where the client is unable to list "private" (chmod 700-ed) directories, even if the current user owns the directory in question.
A bit more background: we're also using Samba 3.5+Winbind to provide authentication and UID/GID mapping against a Windows 2008 R2 domain controller. I've been asked to get Kerberized NFSv4 working, but this problem occurs on mounts both with and without Kerberos, so I won't torture anyone with the Kerberos part. :-)
ID mapping works:
[joeuser@nfsclient ~]$ id joeuser
uid=56055(joeuser) gid=6502(domain users) groups=6502(domain users),1000001(BUILTIN\users)
Mounting directories -o sec=none works:
[joeuser@nfsclient ~]$ sudo mount -t nfs4 -o proto=tcp,port=2049 -o sec=none nfsserver.example.com:/testdir /mnt
[joeuser@nfsclient ~]$ ls -l /mnt
drwx------ 2 user1 domain users 4096 Aug 3 11:43 user1
drwx------ 2 adbinder domain users 4096 Aug 17 15:20 adbinder
drwx------ 2 joeuser domain users 4096 Aug 3 11:43 joeuser
... however, we can't seem to track down why this happens:
[joeuser@nfsclient ~]$ cd /mnt/joeuser
bash: cd: joeuser: Permission denied
We have the same issue when mounting -o sec=krb5p, which yields slightly more interesting debug output[2].
Both hosts are running CentOS 6.3:
$ uname -a ; cat /etc/redhat-release ; rpm -qa | egrep "(samba|winbind|nfs)"
Linux nfsserver 2.6.32-279.2.1.el6.x86_64 #1 SMP Fri Jul 20 01:55:29 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
CentOS release 6.3 (Final)
nfs-utils-1.2.3-26.el6.x86_64
nfs-utils-lib-1.1.5-4.el6.x86_64
samba-client-3.5.10-125.el6.x86_64
samba-common-3.5.10-125.el6.x86_64
samba-winbind-3.5.10-125.el6.x86_64
samba-winbind-clients-3.5.10-125.el6.x86_64
Out of curiosity, I tried the same AD/Winbind/NFSv4 setup on Fedora 17 and had the same results when mounting either -o sec=none or -o sec=krb5/i/p:
$ uname -a ; cat /etc/redhat-release ; rpm -qa | egrep "(samba|winbind|nfs)"
Linux nfsserver-fedora 3.5.2-1.fc17.x86_64 #1 SMP Wed Aug 15 16:09:27 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Fedora release 17 (Beefy Miracle)
libnfsidmap-0.25-3.fc17.x86_64
libnfsidmap-debuginfo-0.25-3.fc17.x86_64
nfs-utils-1.2.6-3.fc17.x86_64
nfs-utils-debuginfo-1.2.6-3.fc17.x86_64
samba-3.6.6-92.fc17.1.x86_64
samba-client-3.6.6-92.fc17.1.x86_64
samba-common-3.6.6-92.fc17.1.x86_64
samba-winbind-3.6.6-92.fc17.1.x86_64
samba-winbind-clients-3.6.6-92.fc17.1.x86_64
So...
* Is this an ID mapping problem of some kind?
Many of the mailing list posts and bug reports that I've read pertaining to this problem usually stem from ID mapping failure, but it seems to be working in our case[4][5].
* Is it possible that this is an AD/Kerberos problem even though we're mounting -o sec=none and Kerberos is supposedly not involved?
A recent linux-nfs mailing list post[0] described a similar problem when mounting, but the fix had no effect for us. (Once again, I tried mounting -o sec=none and -o sec=krb5/i/p).
* Since we're relying in W2K8R2 for ID mapping, is there something extra we need to do to our least-privilege bind account[3]?
We would greatly appreciate any advice that you NFSv4 experts out there could offer. I would be happy to pass along more details/logs/debug/puzzle pieces if needed.
Many thanks,
--
Derek Warren, IT Services, Research Computing Group, Simon Fraser University
[0] http://www.spinics.net/lists/linux-nfs/msg27440.html
[1] Preamble to mounting. Again, the same problem happens with or without Kerberos.
# net ads join createupn="nfs/nfsserver.example.com@xxxxxxxxxxxxxx" createcomputer="OU" -U $BINDUSER%$BINDPASSWD
# kinit -k NFSSERVER$ (and NFSCLIENT$, but we're going to mount -o sec=none as mentioned above)
# net ads keytab add nfs
# klist -ke
Keytab name: WRFILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
3 host/nfsserver.example.com@xxxxxxxxxxxxxx (des-cbc-crc)
3 host/nfsserver.example.com@xxxxxxxxxxxxxx (des-cbc-md5)
3 host/nfsserver.example.com@xxxxxxxxxxxxxx (arcfour-hmac)
3 host/NFSSERVER@xxxxxxxxxxxxxx (des-cbc-crc)
3 host/NFSSERVER@xxxxxxxxxxxxxx (des-cbc-md5)
3 host/NFSSERVER@xxxxxxxxxxxxxx (arcfour-hmac)
3 NFSSERVER$@AD.EXAMPLE.COM (des-cbc-crc)
3 NFSSERVER$@AD.EXAMPLE.COM (des-cbc-md5)
3 NFSSERVER$@AD.EXAMPLE.COM (arcfour-hmac)
3 nfs/nfsserver.example.com@xxxxxxxxxxxxxx (des-cbc-crc)
3 nfs/nfsserver.example.com@xxxxxxxxxxxxxx (des-cbc-md5)
3 nfs/nfsserver.example.com@xxxxxxxxxxxxxx (arcfour-hmac)
3 nfs/NFSSERVER@xxxxxxxxxxxxxx (des-cbc-crc)
3 nfs/NFSSERVER@xxxxxxxxxxxxxx (des-cbc-md5)
3 nfs/NFSSERVER@xxxxxxxxxxxxxx (arcfour-hmac)
(same `klist -ke` output on the client, too)
[2] The syslog output from the same problem, but when mounted -o sec=krb5/i/p:
nfsclient rpc.gssd[10256]: handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt1b)
nfsclient rpc.gssd[10256]: handle_gssd_upcall: 'mech=krb5 uid=56055 enctypes=18,17,16,23,3,1,2 '
nfsclient rpc.gssd[10256]: handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt1b)
nfsclient rpc.gssd[10256]: process_krb5_upcall: service is '<null>'
nfsclient rpc.gssd[10256]: getting credentials for client with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_od8D5s' being considered, with preferred realm 'AD.EXAMPLE.COM'
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_od8D5s'(joeuser@xxxxxxxxxxxxxx) passed all checks and has mtime of 1344544593
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_DUEN2I' being considered, with preferred realm 'AD.EXAMPLE.COM'
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_DUEN2I'(joeuser@xxxxxxxxxxxxxx) passed all checks and has mtime of 1344543679
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_56055_od8D5s' is our current best match with mtime of 1344544593
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_0' being considered, with preferred realm 'AD.EXAMPLE.COM'
nfsclient rpc.gssd[10256]: CC file '/tmp/krb5cc_0' owned by 0, not 56055
nfsclient rpc.gssd[10256]: using FILE:/tmp/krb5cc_56055_od8D5s as credentials cache for client with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: using environment variable to select krb5 ccache FILE:/tmp/krb5cc_56055_od8D5s
nfsclient rpc.gssd[10256]: creating context using fsuid 56055 (save_uid 0)
nfsclient rpc.gssd[10256]: creating tcp client for server nfsserver.example.com
nfsclient rpc.gssd[10256]: DEBUG: port already set to 2049
nfsclient rpc.gssd[10256]: creating context with server nfs@xxxxxxxxxxxxxxxxxxxxx
nfsclient rpc.gssd[10256]: WARNING: Failed to create krb5 context for user with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: WARNING: Failed to create krb5 context for user with uid 56055 for server nfsserver.example.com
nfsclient rpc.gssd[10256]: doing error downcall
[3] Our minimal-privilege Active Directory binding account for Winbind was created using the following steps:
1. Run ADU&C. View menu=>Advanced Features
2. Create a binding account (e.g., 'megabind')
3. Right-click on the FAS container=>Properties=>Security tab
4. [Advanced]=>[Add...]=>megabind=>[OK]
5. [megabind: This object and all descendant objects...]
=> Write All Properties: allow
=> Create Computer objects: allow
6. [OK]/[OK]/[OK]
[4] rpc.idmapd debug output when listing /mnt--same results when mounted -o sec=none/-o sec=krb5/i/p:
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: Server : (user) id "56055" -> name "joeuser@xxxxxxxxxxxxxx"
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfsdcb: authbuf=199.60.0.0/255.255.240.0 authtype=group
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfs4_gid_to_name: calling nsswitch->gid_to_name
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfs4_gid_to_name: nsswitch->gid_to_name returned 0
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfs4_gid_to_name: final return value is 0
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: Server : (group) id "6502" -> name "domain users@xxxxxxxxxxxxxx"
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfsdcb: authbuf=199.60.0.0/255.255.240.0 authtype=user
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfs4_uid_to_name: calling nsswitch->uid_to_name
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfs4_uid_to_name: nsswitch->uid_to_name returned 0
Aug 17 11:28:30 nfsserver rpc.idmapd[1484]: nfs4_uid_to_name: final return value is 0
[5] nfsd debug output:
[joeuser@nfsclient /mnt] cd /mnt/joeuser
Aug 17 11:28:30 nfsclient kernel: encode_compound: tag=
Aug 17 11:28:30 nfsclient kernel: decode_attr_type: type=040000
Aug 17 11:28:30 nfsclient kernel: decode_attr_change: change attribute=17878195753850634499
Aug 17 11:28:30 nfsclient kernel: decode_attr_size: file size=4096
Aug 17 11:28:30 nfsclient kernel: decode_attr_fsid: fsid=(0x64e7963b2d1f48aa/0x98fd26d8b8883c44)
Aug 17 11:28:30 nfsclient kernel: decode_attr_fileid: fileid=2117406
Aug 17 11:28:30 nfsclient kernel: decode_attr_fs_locations: fs_locations done, error = 0
Aug 17 11:28:30 nfsclient kernel: decode_attr_mode: file mode=0755
Aug 17 11:28:30 nfsclient kernel: decode_attr_nlink: nlink=6
Aug 17 11:28:30 nfsclient kernel: decode_attr_owner: uid=0
Aug 17 11:28:30 nfsclient kernel: decode_attr_group: gid=0
Aug 17 11:28:30 nfsclient kernel: decode_attr_rdev: rdev=(0x0:0x0)
Aug 17 11:28:30 nfsclient kernel: decode_attr_space_used: space used=4096
Aug 17 11:28:30 nfsclient kernel: decode_attr_time_access: atime=1345215537
Aug 17 11:28:30 nfsclient kernel: decode_attr_time_metadata: ctime=1344019704
Aug 17 11:28:30 nfsclient kernel: decode_attr_time_modify: mtime=1344019352
Aug 17 11:28:30 nfsclient kernel: decode_attr_mounted_on_fileid: fileid=0
Aug 17 11:28:30 nfsclient kernel: decode_getfattr_attrs: xdr returned 0
Aug 17 11:28:30 nfsclient kernel: decode_getfattr_generic: xdr returned 0
Aug 17 11:28:30 nfsclient kernel: NFS: nfs_update_inode(0:17/2117406 ct=2 info=0x27e7f)
Aug 17 11:28:30 nfsclient kernel: NFS: permission(0:17/2117406), mask=0x1, res=0
Aug 17 11:28:30 nfsclient kernel: NFS: nfs_lookup_revalidate(/joeuser) is valid
Aug 17 11:28:30 nfsclient kernel: encode_compound: tag=
joeuser: Permission denied.
Aug 17 11:28:30 nfsclient kernel: decode_attr_type: type=040000
Aug 17 11:28:30 nfsclient kernel: decode_attr_change: change attribute=8802615669730494470
Aug 17 11:28:30 nfsclient kernel: decode_attr_size: file size=4096
Aug 17 11:28:30 nfsclient kernel: decode_attr_fsid: fsid=(0x64e7963b2d1f48aa/0x98fd26d8b8883c44)
Aug 17 11:28:30 nfsclient kernel: decode_attr_fileid: fileid=2121737
Aug 17 11:28:30 nfsclient kernel: decode_attr_fs_locations: fs_locations done, error = 0
Aug 17 11:28:30 nfsclient kernel: decode_attr_mode: file mode=0700
Aug 17 11:28:30 nfsclient kernel: decode_attr_nlink: nlink=2
Aug 17 11:28:30 nfsclient kernel: decode_attr_owner: uid=56055
Aug 17 11:28:30 nfsclient kernel: decode_attr_group: gid=6502
Aug 17 11:28:30 nfsclient kernel: decode_attr_rdev: rdev=(0x0:0x0)
Aug 17 11:28:30 nfsclient kernel: decode_attr_space_used: space used=4096
Aug 17 11:28:30 nfsclient kernel: decode_attr_time_access: atime=1345071487
Aug 17 11:28:30 nfsclient kernel: decode_attr_time_metadata: ctime=1345071482
Aug 17 11:28:30 nfsclient kernel: decode_attr_time_modify: mtime=1344019451
Aug 17 11:28:30 nfsclient kernel: decode_attr_mounted_on_fileid: fileid=0
Aug 17 11:28:30 nfsclient kernel: decode_getfattr_attrs: xdr returned 0
Aug 17 11:28:30 nfsclient kernel: decode_getfattr_generic: xdr returned 0
Aug 17 11:28:30 nfsclient kernel: NFS: nfs_update_inode(0:17/2121737 ct=1 info=0x27e7f)
Aug 17 11:28:30 nfsclient kernel: NFS: permission(0:17/2121737), mask=0x11, res=-13
Aug 17 11:28:30 nfsclient kernel: NFS: dentry_delete(/joeuser, 8)
Aug 17 11:28:30 nfsclient kernel: NFS: permission(0:17/2117406), mask=0x1, res=0
Aug 17 11:28:30 nfsclient kernel: NFS: nfs_lookup_revalidate(/joeuser) is valid
Aug 17 11:28:30 nfsclient kernel: NFS: permission(0:17/2121737), mask=0x11, res=-13
Aug 17 11:28:30 nfsclient kernel: NFS: dentry_delete(/joeuser, 8)
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
