Hi,
we saw this on a Nautilus cluster when Clients were updated so we had
to modify the client caps to allow read access for the "/" directory.
There's an excerpt in the SUSE docs [1] for that:
If clients with path restriction are used, the MDS capabilities need
to include read access to the root directory.
The allow r path=/ part means that path-restricted clients are able
to see the root volume, but cannot write to it. This may be an issue
for use cases where complete isolation is a requirement.
Can you update the caps and test again?
Regards,
Eugen
[1] https://documentation.suse.com/ses/7.1/html/ses-all/cha-ceph-cephfs.html
Zitat von Wyll Ingersoll <wyllys.ingersoll@xxxxxxxxxxxxxx>:
ceph pacific 16.2.11 (cephadm managed)
I have configured some NFS mounts from the ceph GUI from cephfs. We
can mount the filesystems and view file/directory listings, but
cannot read any file data.
The permissions on the shares are RW. We mount from the client
using "vers=4.1".
Looking at debug logs from the container running nfs-ganesha, I see
the following errors when trying to read a file's content:
15/03/2023 15:27:13 : epoch 6411e209 : gw01 : ganesha.nfsd-7[svc_8]
complete_op :NFS4 :DEBUG :Status of OP_READ in position 2 =
NFS4ERR_PERM, op response size is 7480 total response size is 7568
15/03/2023 15:27:13 : epoch 6411e209 : gw01 : ganesha.nfsd-7[svc_8]
complete_nfs4_compound :NFS4 :DEBUG :End status = NFS4ERR_PERM
lastindex = 3
Also, watching the TCP traffic, I see errors in the NFS protocol
corresponding to these messages:
11:44:43.745570 IP xxx.747 > gw01.nfs: Flags [P.], seq
24184536:24184748, ack 11409577, win 602, options [nop,nop,TS val
342245425 ecr 2683489461], length 212: NFS request xid 156024373 208
getattr fh 0,1/53
11:44:43.745683 IP gw01.nfs > xxx.747: Flags [P.], seq
11409577:11409677, ack 24184748, win 3081, options [nop,nop,TS val
2683489461 ecr 342245425], length 100: NFS reply xid 156024373 reply
ok 96 getattr ERROR: Operation not permitted
So there appears to be a permissions problem where nfs-ganesha is
not able to "getattr" on cephfs data.
The export looks like this (read from rados):
EXPORT {
FSAL {
name = "CEPH";
user_id = "nfs.cephfs.7";
filesystem = "cephfs";
secret_access_key = "xxx";
}
export_id = 7;
path = "/exports/nfs/foobar";
pseudo = "/foobar";
access_type = "RW";
squash = "no_root_squash";
attr_expiration_time = 0;
security_label = false;
protocols = 4;
transports = "TCP";
}
ceph auth permissions for the nfs.cephfs.7 client:
[client.nfs.cephfs.7]
key = xxx
caps mds = "allow rw path=/exports/nfs/foobar"
caps mon = "allow r"
caps osd = "allow rw pool=.nfs namespace=cephfs, allow rw tag
cephfs data=cephfs"
Any suggestions?
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
