Re: Cephfs issue - able to mount with user key, not able to write

[Goncalo Borges <goncalo.borges@xxxxxxxxxxxxx>]

Hi ...

We also use a mount_user key to mount cephfs with ceph-fuse. I remember that we had some troubles also. We use ceph-authtool to generate the key with the following syntax:

   ceph-authtool --create-keyring <path to store the keyring>
                 --gen-key -n <username>
                 --cap mds '<permissions>'
                 --cap osd '<permissions> pool=<poolname>'
                 --cap mon '<permissions>'

In our case, the command we executed was

# ceph-authtool --create-keyring /etc/ceph/ceph.client.mount_user.keyring --gen-key -n client.mount_user --cap mds 'allow' --cap osd 'allow rw pool=coepp_cephfs_data' --cap mon 'allow r

Please note the following particularities:
   - The name of the key should be <clustername>.client.<username>.keyring (i.e. ceph.client.mount_user.keyring)
   - The name of the user should be client.<username> (i.e. client.mount_user)
   - This key has the following permissions:
     --cap mds 'allow'
     --cap osd 'allow rw pool=coepp_cephfs_data'
     --cap mon 'allow r'

It seems you are following this structure but please double check it. I also remember some bugs in Jewel regarding permissions in paths. So you might think to remote it from the key. After creation, one should import the key to the authority list

# ceph auth import -i /etc/ceph/ceph.client.mount_user.keyring

# ceph auth list
installed auth entries:

(...)

client.mount_user
        key: ....
        caps: [mds] allow
        caps: [mon] allow r
        caps: [osd] allow rw pool=coepp_cephfs_data

Finally I mount it as

# ceph-fuse --id mount_user -k /etc/ceph/ceph.client.mount_user.keyring -m X.X.X.X:6789 -r /cephfs /coepp/cephfs/'

where X.X.X.X stands for the mom ip address.

Does this help?
Cheers
G.

On 08/03/2016 06:23 PM, Daleep Singh Bais wrote:

Dear All,

I am trying to use CephFS in my setup. I have created a test setup with
01 MON and 03 OSD's.

I have created a MDS server and able to mount it on client using FUSE.
Using admin keyring, I am able to write to cephfs and sub-dir also.

I am experiencing issue when I try to write to cephfs using another
user. I have created the reqd keys with permissions.

When I try to write, I see that metadata object count increase, but no
change in data pool.

Also this is what i see in logs

2016-08-03 08:17:20.771597 b16feff0  0 log_channel(cluster) log [INF] :
closing stale session client.165552 192.168.1.29:0/5671 after 302.321097
2016-08-03 08:19:16.049985 accfeff0  0 -- 192.168.1.201:6800/7088 >>
192.168.1.29:0/5707 pipe(0x8549ed00 sd=22 :6800 s=2 pgs=2 cs=1 l=0
c=0x857342e0).fault with nothing to send, going to standby

My cephx key is client.test1 created like :

ceph auth get-or-create client.test1 mon 'allow r' mds 'allow r, allow
rw path=/test1' osd 'allow rw pool=data' -o
/etc/ceph/ceph.client.test1.keyring

#ceph mds stat
e11: 1/1/1 up {0=mon1=up:active}

# ceph --version
ceph version 10.2.2-1-g502540f (502540faf67308fa595e03f9f446b4ba67df731d)


Any suggestion would be helpful.

Thanks.

Daleep Singh Bais

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

-- 
Goncalo Borges
Research Computing
ARC Centre of Excellence for Particle Physics at the Terascale
School of Physics A28 | University of Sydney, NSW  2006
T: +61 2 93511937

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com