Ceph Meta-data Server (MDS) installation giving error

[prasad pande <pande.prasad1@xxxxxxxxx>]

Hi All,

I installed the Ceph cluster on CentOS using the following configuration file:

[global]
fsid = f1fccdcf-f53f-4a6d-93f2-002f392e98da
mon_initial_members = ceph-mon1
mon_host = 192.168.122.213
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true
#public network = 192.168.122.10/256

#[global]
#auth_service_required = cephx
#auth_client_required = cephx
#auth_cluster_required = cephx

[mon.ceph-mon1]
mon_addr = 192.168.122.213:6789
host = ceph-mon1

[osd]
filestore_xattr_use_omap = true
osd_data = /var/lib/ceph/osd/$cluster-$id
osd_journal_size = 1024

#[mds]
#mds data = "">#keyring = /etc/ceph/keyring.$id

[mds.ceph-mds2]
host = ceph-mds2

My current ceph status is as follows:

[root@ceph-mon1 ceph]# ceph status
    cluster f1fccdcf-f53f-4a6d-93f2-002f392e98da
     health HEALTH_WARN
            30 pgs degraded
            30 pgs stuck degraded
            80 pgs stuck unclean
            30 pgs stuck undersized
            30 pgs undersized
            recovery 5/48 objects degraded (10.417%)
            recovery 11/48 objects misplaced (22.917%)
     monmap e1: 1 mons at {ceph-mon1=192.168.122.213:6789/0}
            election epoch 1, quorum 0 ceph-mon1
     osdmap e63: 4 osds: 4 up, 4 in; 50 remapped pgs
      pgmap v1842: 80 pgs, 3 pools, 1032 MB data, 16 objects
            2210 MB used, 123 GB / 125 GB avail
            5/48 objects degraded (10.417%)
            11/48 objects misplaced (22.917%)
                  50 active+remapped
                  30 active+undersized+degraded

Now I am trying to add new MDS in the server and I am getting and authentication error as follows:

[root@ceph-mon1 ceph]# ceph-deploy --overwrite-conf mds create ceph-mds2
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.28): /usr/bin/ceph-deploy --overwrite-conf mds create ceph-mds2
[ceph_deploy.cli][INFO  ] ceph-deploy options:
[ceph_deploy.cli][INFO  ]  username                      : None
[ceph_deploy.cli][INFO  ]  verbose                       : False
[ceph_deploy.cli][INFO  ]  overwrite_conf                : True
[ceph_deploy.cli][INFO  ]  subcommand                    : create
[ceph_deploy.cli][INFO  ]  quiet                         : False
[ceph_deploy.cli][INFO  ]  cd_conf                       : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fb646f40170>
[ceph_deploy.cli][INFO  ]  cluster                       : ceph
[ceph_deploy.cli][INFO  ]  func                          : <function mds at 0x7fb6477ce8c0>
[ceph_deploy.cli][INFO  ]  ceph_conf                     : None
[ceph_deploy.cli][INFO  ]  mds                           : [('ceph-mds2', 'ceph-mds2')]
[ceph_deploy.cli][INFO  ]  default_release               : False
[ceph_deploy.mds][DEBUG ] Deploying mds, cluster ceph hosts ceph-mds2:ceph-mds2
root@ceph-mds2's password:
root@ceph-mds2's password:
[ceph-mds2][DEBUG ] connected to host: ceph-mds2
[ceph-mds2][DEBUG ] detect platform information from remote host
[ceph-mds2][DEBUG ] detect machine type
[ceph_deploy.mds][INFO  ] Distro info: CentOS Linux 7.1.1503 Core
[ceph_deploy.mds][DEBUG ] remote host will use sysvinit
[ceph_deploy.mds][DEBUG ] deploying mds bootstrap to ceph-mds2
[ceph-mds2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph-mds2][DEBUG ] create path if it doesn't exist
[ceph-mds2][INFO  ] Running command: ceph --cluster ceph --name client.bootstrap-mds --keyring /var/lib/ceph/bootstrap-mds/ceph.keyring auth get-or-create mds.ceph-mds2 osd allow rwx mds allow mon allow profile mds -o /var/lib/ceph/mds/ceph-ceph-mds2/keyring
[ceph-mds2][ERROR ] 2015-11-13 21:56:46.262122 7f3077653700  0 librados: client.bootstrap-mds authentication error (1) Operation not permitted
[ceph-mds2][ERROR ] Error connecting to cluster: PermissionError
[ceph-mds2][ERROR ] exit code from command was: 1
[ceph_deploy.mds][ERROR ] could not create mds
[ceph_deploy][ERROR ] GenericError: Failed to create 1 MDSs

I client-admin.keyring on the new mds server as follows:

[root@ceph-mon1 ceph]# scp ceph.client.admin.keyring ceph-mds2:/etc/ceph
[root@ceph-mon1 ceph]# scp /var/lib/ceph/bootstrap-osd/ceph.keyring ceph-mds2:/var/lib/ceph/bootstrap-osd
[root@ceph-mon1 ceph]# scp /var/lib/ceph/bootstrap-mds/ceph.keyring ceph-mds2:/var/lib/ceph/bootstrap-mds

But after doing this also, it is giving me the same error.

Can anyone guide me for this issue?

Thanks in advance.

Regards,

Prasad Pande

Thanks & Regards
 
Prasad Pande

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com