Hi,
I've encountered some problems accesing files on CephFS:
$ ls -al syntenyPlot.png
-rw-r----- 1 edgar edgar 9329 Jun 11 2014 syntenyPlot.png
$ groups
... edgar ...
$ cat syntenyPlot.png
cat: syntenyPlot.png: Permission denied
CephFS is mounted via ceph-fuse:
ceph-fuse on /ceph type fuse.ceph-fuse
(rw,nosuid,nodev,noatime,allow_other,default_permissions)
OS is Ubuntu 14.04, Ceph version is 0.94.2
I've isolated a test machine and activated debugging (debug_client =
20/20). The following lines correspond to the 'cat' invocation:
2015-08-04 12:59:44.030372 7f574dffb700 20 client.421984 _ll_get
0x7f5758024da0 100022310be -> 13
2015-08-04 12:59:44.030398 7f574dffb700 3 client.421984 ll_getattr
100022310be.head
2015-08-04 12:59:44.030403 7f574dffb700 10 client.421984 _getattr mask
pAsLsXsFs issued=1
2015-08-04 12:59:44.030413 7f574dffb700 10 client.421984 fill_stat on
100022310be snap/devhead mode 042770 mtime 2014-06-12 09:31:39.000000
ctime 2015-07-31 14:17:12.364416
2015-08-04 12:59:44.030426 7f574dffb700 3 client.421984 ll_getattr
100022310be.head = 0
2015-08-04 12:59:44.030443 7f574dffb700 3 client.421984 ll_forget
100022310be 1
2015-08-04 12:59:44.030447 7f574dffb700 20 client.421984 _ll_put
0x7f5758024da0 100022310be 1 -> 12
2015-08-04 12:59:44.030459 7f574dffb700 20 client.421984 _ll_get
0x7f5758024da0 100022310be -> 13
2015-08-04 12:59:44.030463 7f574dffb700 3 client.421984 ll_lookup
0x7f5758024da0 syntenyPlot.png
2015-08-04 12:59:44.030469 7f574dffb700 20 client.421984 _lookup have dn
syntenyPlot.png mds.-1 ttl 0.000000 seq 0
2015-08-04 12:59:44.030476 7f574dffb700 10 client.421984 _lookup
100022310be.head(ref=3 ll_ref=13 cap_refs={} open={} mode=42770 size=0/0
mtime=2014-06-12 09:31:39.000000 caps=pAsLsXsFs(0=pAsLsXsFs) COMPLETE
parents=0x7f57580261d0 0x7f5758024da0) syntenyPlot.png =
1000223121e.head(ref=2 ll_ref=20 cap_refs={} open={} mode=100640
size=9329/0 mtime=2014-06-11 09:05:47.000000
caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[1000223121e ts 0/0 objects 0
dirty_or_tx 0] parents=0x7f575802d290 0x7f575802c5a0)
2015-08-04 12:59:44.030530 7f574dffb700 10 client.421984 fill_stat on
1000223121e snap/devhead mode 0100640 mtime 2014-06-11 09:05:47.000000
ctime 2015-08-04 11:07:53.623370
2015-08-04 12:59:44.030539 7f574dffb700 20 client.421984 _ll_get
0x7f575802c5a0 1000223121e -> 21
2015-08-04 12:59:44.030542 7f574dffb700 3 client.421984 ll_lookup
0x7f5758024da0 syntenyPlot.png -> 0 (1000223121e)
2015-08-04 12:59:44.030555 7f574dffb700 3 client.421984 ll_forget
100022310be 1
2015-08-04 12:59:44.030558 7f574dffb700 20 client.421984 _ll_put
0x7f5758024da0 100022310be 1 -> 12
2015-08-04 12:59:44.030628 7f57467fc700 20 client.421984 _ll_get
0x7f575802c5a0 1000223121e -> 22
2015-08-04 12:59:44.030645 7f57467fc700 3 client.421984 ll_getattr
1000223121e.head
2015-08-04 12:59:44.030649 7f57467fc700 10 client.421984 _getattr mask
pAsLsXsFs issued=1
2015-08-04 12:59:44.030659 7f57467fc700 10 client.421984 fill_stat on
1000223121e snap/devhead mode 0100640 mtime 2014-06-11 09:05:47.000000
ctime 2015-08-04 11:07:53.623370
2015-08-04 12:59:44.030672 7f57467fc700 3 client.421984 ll_getattr
1000223121e.head = 0
2015-08-04 12:59:44.030690 7f57467fc700 3 client.421984 ll_forget
1000223121e 1
2015-08-04 12:59:44.030695 7f57467fc700 20 client.421984 _ll_put
0x7f575802c5a0 1000223121e 1 -> 21
2015-08-04 12:59:44.030760 7f574e7fc700 20 client.421984 _ll_get
0x7f575802c5a0 1000223121e -> 22
2015-08-04 12:59:44.030775 7f574e7fc700 3 client.421984 ll_open
1000223121e.head 32768
2015-08-04 12:59:44.030779 7f574e7fc700 3 client.421984 ll_open
1000223121e.head 32768 = -13 (0)
2015-08-04 12:59:44.030797 7f574e7fc700 3 client.421984 ll_forget
1000223121e 1
2015-08-04 12:59:44.030802 7f574e7fc700 20 client.421984 _ll_put
0x7f575802c5a0 1000223121e 1 -> 21
The return value of -13 in the open call is probably 'permission denied'.
The setup looks ok with respect to permissions. The root user is able to
read the file in question. The owning user is also able to read the file
(after sudo). The problem occurs on several hosts for a number of files,
but not all files or all users on CephFS are affected. User and group
information are stored in LDAP and made available via SSSD; ls -l
displays to correct group and user names, and id(1) lists the correct id
and names.
Any hints on what's going wrong here?
Best regards,
Burkhard
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
