Hello, I've tried to install the ceph using ceph-deploy as usual. [ceph@octopus conf]$ ceph-deploy install --mon --mds --testing octopus *install* was completed without any surprising. But *mon create-initial* was failed: ### Take1 ### #### Log #### [ceph@octopus conf]$ ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.31): /usr/bin/ceph-deploy mon create-initial [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] overwrite_conf : False [ceph_deploy.cli][INFO ] subcommand : create-initial [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7efcb3c2ab00> [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] func : <function mon at 0x7efcb3c24320> [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.cli][INFO ] keyrings : None [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts octopus [ceph_deploy.mon][DEBUG ] detecting platform for host octopus ... [octopus][DEBUG ] connection detected need for sudo [octopus][DEBUG ] connected to host: octopus [octopus][DEBUG ] detect platform information from remote host [octopus][DEBUG ] detect machine type [octopus][DEBUG ] find the location of an executable [ceph_deploy.mon][INFO ] distro info: CentOS Linux 7.2.1511 Core [octopus][DEBUG ] determining if provided host has same hostname in remote [octopus][DEBUG ] get remote short hostname [octopus][DEBUG ] deploying mon to octopus [octopus][DEBUG ] get remote short hostname [octopus][DEBUG ] remote hostname: octopus [octopus][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [octopus][DEBUG ] create the mon path if it does not exist [octopus][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-octopus/done [octopus][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-octopus/done [octopus][INFO ] creating keyring file: /var/lib/ceph/tmp/ceph-octopus.mon.keyring [octopus][DEBUG ] create the monitor keyring file [octopus][INFO ] Running command: sudo ceph-mon --cluster ceph --mkfs -i octopus --keyring /var/lib/ceph/tmp/ceph-octopus.mon.keyring --setuser 1000 --setgroup 1000 [octopus][DEBUG ] ceph-mon: mon.noname-a 172.16.0.2:6789/0 is local, renaming to mon.octopus [octopus][DEBUG ] ceph-mon: set fsid to 53dab59e-3e16-4fdf-a029-b92c32aabde8 [octopus][DEBUG ] ceph-mon: created monfs at /var/lib/ceph/mon/ceph-octopus for mon.octopus [octopus][INFO ] unlinking keyring file /var/lib/ceph/tmp/ceph-octopus.mon.keyring [octopus][DEBUG ] create a done file to avoid re-doing the mon deployment [octopus][DEBUG ] create the init path if it does not exist [octopus][INFO ] Running command: sudo systemctl enable ceph.target [octopus][WARNIN] Failed to execute operation: Access denied [octopus][ERROR ] RuntimeError: command returned non-zero exit status: 1 [ceph_deploy.mon][ERROR ] Failed to execute command: systemctl enable ceph.target [ceph_deploy][ERROR ] GenericError: Failed to create 1 monitors #### Log #### Interestingly after rebooting the host, it's completed -; ### Take2 ### #### Log #### [ceph@octopus conf]$ ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.31): /usr/bin/ceph-deploy mon create-initial [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] overwrite_conf : False [ceph_deploy.cli][INFO ] subcommand : create-initial [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f8cc1978b00> [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] func : <function mon at 0x7f8cc1972320> [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.cli][INFO ] keyrings : None [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts octopus [ceph_deploy.mon][DEBUG ] detecting platform for host octopus ... [octopus][DEBUG ] connection detected need for sudo [octopus][DEBUG ] connected to host: octopus [octopus][DEBUG ] detect platform information from remote host [octopus][DEBUG ] detect machine type [octopus][DEBUG ] find the location of an executable [ceph_deploy.mon][INFO ] distro info: CentOS Linux 7.2.1511 Core [octopus][DEBUG ] determining if provided host has same hostname in remote [octopus][DEBUG ] get remote short hostname [octopus][DEBUG ] deploying mon to octopus [octopus][DEBUG ] get remote short hostname [octopus][DEBUG ] remote hostname: octopus [octopus][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [octopus][DEBUG ] create the mon path if it does not exist [octopus][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-octopus/done [octopus][DEBUG ] create a done file to avoid re-doing the mon deployment [octopus][DEBUG ] create the init path if it does not exist [octopus][INFO ] Running command: sudo systemctl enable ceph.target [octopus][INFO ] Running command: sudo systemctl enable ceph-mon@octopus [octopus][INFO ] Running command: sudo systemctl start ceph-mon@octopus [octopus][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.octopus.asok mon_status [octopus][DEBUG ] ******************************************************************************** [octopus][DEBUG ] status for monitor: mon.octopus [octopus][DEBUG ] { [octopus][DEBUG ] "election_epoch": 3, [octopus][DEBUG ] "extra_probe_peers": [], [octopus][DEBUG ] "monmap": { [octopus][DEBUG ] "created": "0.000000", [octopus][DEBUG ] "epoch": 1, [octopus][DEBUG ] "fsid": "53dab59e-3e16-4fdf-a029-b92c32aabde8", [octopus][DEBUG ] "modified": "0.000000", [octopus][DEBUG ] "mons": [ [octopus][DEBUG ] { [octopus][DEBUG ] "addr": "172.16.0.2:6789/0", [octopus][DEBUG ] "name": "octopus", [octopus][DEBUG ] "rank": 0 [octopus][DEBUG ] } [octopus][DEBUG ] ] [octopus][DEBUG ] }, [octopus][DEBUG ] "name": "octopus", [octopus][DEBUG ] "outside_quorum": [], [octopus][DEBUG ] "quorum": [ [octopus][DEBUG ] 0 [octopus][DEBUG ] ], [octopus][DEBUG ] "rank": 0, [octopus][DEBUG ] "state": "leader", [octopus][DEBUG ] "sync_provider": [] [octopus][DEBUG ] } [octopus][DEBUG ] ******************************************************************************** [octopus][INFO ] monitor: mon.octopus is running [octopus][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.octopus.asok mon_status [ceph_deploy.mon][INFO ] processing monitor mon.octopus [octopus][DEBUG ] connection detected need for sudo [octopus][DEBUG ] connected to host: octopus [octopus][DEBUG ] detect platform information from remote host [octopus][DEBUG ] detect machine type [octopus][DEBUG ] find the location of an executable [octopus][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.octopus.asok mon_status [ceph_deploy.mon][INFO ] mon.octopus monitor has reached quorum! [ceph_deploy.mon][INFO ] all initial monitors are running and have formed quorum [ceph_deploy.mon][INFO ] Running gatherkeys... [ceph_deploy.gatherkeys][DEBUG ] Checking octopus for /etc/ceph/ceph.client.admin.keyring [octopus][DEBUG ] connection detected need for sudo [octopus][DEBUG ] connected to host: octopus [octopus][DEBUG ] detect platform information from remote host [octopus][DEBUG ] detect machine type [octopus][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][DEBUG ] Got ceph.client.admin.keyring key from octopus. [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring [ceph_deploy.gatherkeys][DEBUG ] Checking octopus for /var/lib/ceph/bootstrap-osd/ceph.keyring [octopus][DEBUG ] connection detected need for sudo [octopus][DEBUG ] connected to host: octopus [octopus][DEBUG ] detect platform information from remote host [octopus][DEBUG ] detect machine type [octopus][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][DEBUG ] Got ceph.bootstrap-osd.keyring key from octopus. [ceph_deploy.gatherkeys][DEBUG ] Checking octopus for /var/lib/ceph/bootstrap-mds/ceph.keyring [octopus][DEBUG ] connection detected need for sudo [octopus][DEBUG ] connected to host: octopus [octopus][DEBUG ] detect platform information from remote host [octopus][DEBUG ] detect machine type [octopus][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][DEBUG ] Got ceph.bootstrap-mds.keyring key from octopus. [ceph_deploy.gatherkeys][DEBUG ] Checking octopus for /var/lib/ceph/bootstrap-rgw/ceph.keyring [octopus][DEBUG ] connection detected need for sudo [octopus][DEBUG ] connected to host: octopus [octopus][DEBUG ] detect platform information from remote host [octopus][DEBUG ] detect machine type [octopus][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][DEBUG ] Got ceph.bootstrap-rgw.keyring key from octopus. #### Log #### ####### [ceph@octopus conf]$ ls -l *.keyring -rw-------. 1 ceph ceph 71 Mar 15 19:13 ceph.bootstrap-mds.keyring -rw-------. 1 ceph ceph 71 Mar 15 19:13 ceph.bootstrap-osd.keyring -rw-------. 1 ceph ceph 71 Mar 15 19:13 ceph.bootstrap-rgw.keyring -rw-------. 1 ceph ceph 63 Mar 15 19:13 ceph.client.admin.keyring -rw-------. 1 ceph ceph 73 Mar 15 19:04 ceph.mon.keyring ####### ## Version ## [ceph@octopus ~]$ ceph-deploy --version 1.5.31 [ceph@octopus conf]$ ceph --version ceph version 10.0.2 (86764eaebe1eda943c59d7d784b893ec8b0c6ff9) Any suggestion would be help for the next. Cheers, Shinobu _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com