Re: My misadventure in trying to install Ceph Emperor

[Ron Gage <ron@xxxxxxxxxxx>]

Mike:

[ceph@cm ~]$ getenforce
Disabled
[ceph@cm ~]$

 

On February 14, 2014 at 11:18 AM "Michael J. Kidd" <michael.kidd@xxxxxxxxxxx> wrote:

I'd also be curious if you happen to have selinux enabled...

$ getenforce
Enforcing

^-- bad for Ceph

If you have this, try:

$ sudo setenforce 0

$ getenforce

Permissive

To make the change stick after reboot, edit /etc/selinux/config and change the SELINUX= line as follows:
SELINUX=permissive

Or
SELINUX=disabled

Either will work.

Michael J. Kidd
Sr. Storage Consultant
Inktank Professional Services

On Fri, Feb 14, 2014 at 11:08 AM, Joao Eduardo Luis <joao.luis@xxxxxxxxxxx> wrote:

On 02/14/2014 03:56 PM, Ron Gage wrote:

Running ceph-create-keys -i cm as ceph results in the following line
being repeated forever:

admin_socket: exception getting command descriptions: [Errno 13]
Permission denied
INFO:ceph-create-keys:ceph-mon admin socket not ready yet.

Running sudo ceph-create-keys -i cm results in the following:

INFO:ceph-create-keys:Talking to monitor...
2014-02-14 15:51:10.330209 7f2aec497700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2ae8024400 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2ae8024660).fault

This looks like the tools are attempting to contact the monitor at 10.0.0.6:6789, while the monitor believes to be at 172.24.12.91:6789 (from one of your previous emails to the list).

I would suggest two things.

1) try running 'ceph -s -m 172.24.12.91:6789' and 'ceph -s -m 10.0.0.6:6789' and see if any of the two complete.

2) If above fails, crank up the monitor debugging (debug mon = 10, debug ms = 1) and send a log on our way so we can take a look.

  -Joao

2014-02-14 15:51:13.330386 7f2aec396700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc000c00 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc000e60).fault
2014-02-14 15:51:16.330809 7f2aec497700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc003010 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc003270).fault
2014-02-14 15:51:19.331141 7f2aec396700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc0036b0 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc003910).fault
2014-02-14 15:51:22.331725 7f2aec497700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc0025a0 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc000cd0).fault
2014-02-14 15:51:25.331912 7f2aec396700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc002ae0 sd=4 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc002d40).fault
2014-02-14 15:51:28.332318 7f2aec497700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc003e80 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc0040e0).fault
2014-02-14 15:51:31.332823 7f2aec396700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc0046c0 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc004920).fault
2014-02-14 15:51:34.333134 7f2aec497700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc006010 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc006270).fault
2014-02-14 15:51:37.333430 7f2aec396700  0 -- :/1002299 >>
10.0.0.6:6789/0 pipe(0x7f2adc0063d0 sd=3 :0 s=1 pgs=0 cs=0 l=1
c=0x7f2adc006630).fault

... and it goes on forever

[ceph@cm my-cluster]$ sudo iptables -L

Chain INPUT (policy ACCEPT) target     prot opt source
destination

Chain FORWARD (policy ACCEPT) target     prot opt source
destination

Chain OUTPUT (policy ACCEPT) target     prot opt source
destination

[ceph@cm my-cluster]$

 > On February 14, 2014 at 10:23 AM Alfredo Deza
< alfredo.deza@xxxxxxxxxxx> wrote:
 >
 >
 > On Fri, Feb 14, 2014 at 10:14 AM, Ron Gage < ron@xxxxxxxxxxx> wrote:
 > > Alfredo:
 > > As near as I can tell, ceph-create-keys is hanging - or taking a
huge amount
 > > ot fime to finish.
 >
 > Correct! Do you have iptables enabled by any chance?
 >
 > Also it can be useful if you manually execute that create-keys command
 > and see the output.
 > >
 > > Segment from screen output:
 > > (Make sure that ALL ceph programs/processes are not running)
 > >
 > > [ceph@cm my-cluster]$ ceph-deploy mon create-initial cm
 > > ...
 > > [cm][DEBUG ] === mon.cm ===
 > > [cm][DEBUG ] Starting Ceph mon.cm on cm...
 > > [cm][DEBUG ] Starting ceph-create-keys on cm...
 > > [cm][WARNIN] No data was received after 7 seconds, disconnecting...
 > > [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon
 > > /var/run/ceph/ceph-mon.cm.asok mon_status
 > > ...
 > >
 > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > > /etc/ceph/ceph.client.admin. keyring
 > > [cm][DEBUG ] connected to host: cm
 > > [cm][DEBUG ] detect platform information from remote host
 > > [cm][DEBUG ] detect machine type
 > > [cm][DEBUG ] fetch remote file
 > > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > > /etc/ceph/ceph.client.admin. keyring on ['cm']
 > > [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
 > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > > /var/lib/ceph/bootstrap-osd/ ceph.keyring
 > > [cm][DEBUG ] connected to host: cm
 > > [cm][DEBUG ] detect platform information from remote host
 > > [cm][DEBUG ] detect machine type
 > > [cm][DEBUG ] fetch remote file
 > > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > > /var/lib/ceph/bootstrap-osd/ ceph.keyring on ['cm']
 > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > > /var/lib/ceph/bootstrap-mds/ ceph.keyring
 > > [cm][DEBUG ] connected to host: cm
 > > [cm][DEBUG ] detect platform information from remote host
 > > [cm][DEBUG ] detect machine type
 > > [cm][DEBUG ] fetch remote file
 > > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > > /var/lib/ceph/bootstrap-mds/ ceph.keyring on ['cm']
 > >
 > > [ceph@cm my-cluster]$ ps ax | grep -i ceph
 > > 1298 ? Ss 0:00 sshd: ceph [priv]
 > > 1302 ? S 0:00 sshd: ceph@pts/0
 > > 2058 pts/0 Sl 0:00 /usr/bin/ceph-mon -i cm --pid-file
 > > /var/run/ceph/mon.cm.pid -c /etc/ceph/ceph.conf
 > > 2071 pts/0 S 0:00 python /usr/sbin/ceph-create-keys -i cm
 > > 2081 pts/0 Sl 0:00 python /usr/bin/ceph --cluster=ceph --name=mon.
 > > --keyring=/var/lib/ceph/mon/ ceph-cm/keyring auth get-or-create
client.admin
 > > mon allow * osd allow * mds allow
 > > 2212 pts/0 S+ 0:00 grep -i ceph
 > > [ceph@cm my-cluster]$
 > >
 > >
 > >
 > >> On February 14, 2014 at 9:41 AM Alfredo Deza
< alfredo.deza@xxxxxxxxxxx>
 > >> wrote:
 > >>
 > >>
 > >> On Fri, Feb 14, 2014 at 9:38 AM, Ron Gage < ron@xxxxxxxxxxx> wrote:
 > >> > Thanks Alfredo!
 > >> > Would that be /etc/ceph/ceph.conf or ~/my-cluster/ceph.conf
 > >>
 > >> Good question, sorry for the ambiguity. From where you are running
 > >> ceph-deploy, you should have a ceph.conf there, you can edit
 > >> that one file and then call 'ceph-deploy config push {nodes}' where
 > >> {nodes} should be replaced with the server(s) to push the config to.
 > >>
 > >>
 > >> >
 > >> > Ron Gage
 > >> >
 > >> >
 > >> >> On February 14, 2014 at 9:31 AM Alfredo Deza
< alfredo.deza@xxxxxxxxxxx>
 > >> >> wrote:
 > >> >>
 > >> >>
 > >> >> On Fri, Feb 14, 2014 at 9:08 AM, Ron Gage < ron@xxxxxxxxxxx> wrote:
 > >> >> > Please see the post you replied to. The admin keyring is not
being
 > >> >> > created.
 > >> >> >
 > >> >> > Thanks!
 > >> >> >
 > >> >> > Ron
 > >> >>
 > >> >> There is no need to run other commands instead of
create-initial. That
 > >> >> is the best way to get started with monitor deployment.
 > >> >>
 > >> >> From ceph-deploy's perspective, everything has been called
correctly
 > >> >> and the output from the node looks correct. The next thing I would
 > >> >> suggest is to bump the log levels of the monitors in ceph.conf and
 > >> >> restart them so that you can inspect a bit better what is going on
 > >> >> there.
 > >> >>
 > >> >> In the [global] section of ceph.conf:
 > >> >>
 > >> >> mon debug = 10
 > >> >>
 > >> >>
 > >> >>
 > >> >> >
 > >> >> >
 > >> >> > On February 14, 2014 at 8:54 AM Srinivasa Rao Ragolu
 > >> >> > < sragolu@xxxxxxxxxx>
 > >> >> > wrote:
 > >> >> >
 > >> >> > Follow the below link as it is
 > >> >> >
 > >> >> >
http://karan-mj.blogspot.fi/2013/12/ceph-installation-part-2.html
 > >> >> >
 > >> >> > Thanks,
 > >> >> > Srinivas.
 > >> >> >
 > >> >> >
 > >> >> > On Fri, Feb 14, 2014 at 7:21 PM, Ron Gage < ron@xxxxxxxxxxx>
wrote:
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > [ceph@cm my-cluster]$ ceph-deploy mon create cm
 > >> >> > [ceph_deploy.cli][INFO ] Invoked (1.3.5):
/usr/bin/ceph-deploy mon
 > >> >> > create
 > >> >> > cm
 > >> >> > [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts cm
 > >> >> > [ceph_deploy.mon][DEBUG ] detecting platform for host cm ...
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [ceph_deploy.mon][INFO ] distro info: CentOS 6.5 Final
 > >> >> > [cm][DEBUG ] determining if provided host has same hostname
in remote
 > >> >> > [cm][DEBUG ] get remote short hostname
 > >> >> > [cm][DEBUG ] deploying mon to cm
 > >> >> > [cm][DEBUG ] get remote short hostname
 > >> >> > [cm][DEBUG ] remote hostname: cm
 > >> >> > [cm][DEBUG ] write cluster configuration to
/etc/ceph/{cluster}.conf
 > >> >> > [cm][DEBUG ] create the mon path if it does not exist
 > >> >> > [cm][DEBUG ] checking for done path:
/var/lib/ceph/mon/ceph-cm/done
 > >> >> > [cm][DEBUG ] create a done file to avoid re-doing the mon
deployment
 > >> >> > [cm][DEBUG ] create the init path if it does not exist
 > >> >> > [cm][DEBUG ] locating the `service` executable...
 > >> >> > [cm][INFO ] Running command: sudo /sbin/service ceph -c
 > >> >> > /etc/ceph/ceph.conf
 > >> >> > start mon.cm
 > >> >> > [cm][DEBUG ] === mon.cm ===
 > >> >> > [cm][DEBUG ] Starting Ceph mon.cm on cm...
 > >> >> > [cm][DEBUG ] Starting ceph-create-keys on cm...
 > >> >> > [cm][WARNIN] No data was received after 7 seconds,
disconnecting...
 > >> >> > [cm][INFO ] Running command: sudo ceph --cluster=ceph
--admin-daemon
 > >> >> > /var/run/ceph/ceph-mon.cm.asok mon_status
 > >> >> > [cm][DEBUG ]
 > >> >> >
 > >> >> >
 > >> >> >
****************************** ****************************** ********************

 > >> >> > [cm][DEBUG ] status for monitor: mon.cm
 > >> >> > [cm][DEBUG ] {
 > >> >> > [cm][DEBUG ] "election_epoch": 1,
 > >> >> > [cm][DEBUG ] "extra_probe_peers": [],
 > >> >> > [cm][DEBUG ] "monmap": {
 > >> >> > [cm][DEBUG ] "created": "0.000000",
 > >> >> > [cm][DEBUG ] "epoch": 1,
 > >> >> > [cm][DEBUG ] "fsid": "b11875f0-e611-40b9-ae81- e7cba7586ed5",
 > >> >> > [cm][DEBUG ] "modified": "0.000000",
 > >> >> > [cm][DEBUG ] "mons": [
 > >> >> > [cm][DEBUG ] {
 > >> >> > [cm][DEBUG ] "addr": " 172.24.12.91:6789/0",
 > >> >> > [cm][DEBUG ] "name": "cm",
 > >> >> > [cm][DEBUG ] "rank": 0
 > >> >> > [cm][DEBUG ] }
 > >> >> > [cm][DEBUG ] ]
 > >> >> > [cm][DEBUG ] },
 > >> >> > [cm][DEBUG ] "name": "cm",
 > >> >> > [cm][DEBUG ] "outside_quorum": [],
 > >> >> > [cm][DEBUG ] "quorum": [
 > >> >> > [cm][DEBUG ] 0
 > >> >> > [cm][DEBUG ] ],
 > >> >> > [cm][DEBUG ] "rank": 0,
 > >> >> > [cm][DEBUG ] "state": "leader",
 > >> >> > [cm][DEBUG ] "sync_provider": []
 > >> >> > [cm][DEBUG ] }
 > >> >> > [cm][DEBUG ]
 > >> >> >
 > >> >> >
 > >> >> >
****************************** ****************************** ********************

 > >> >> > [cm][INFO ] monitor: mon.cm is running
 > >> >> > [cm][INFO ] Running command: sudo ceph --cluster=ceph
--admin-daemon
 > >> >> > /var/run/ceph/ceph-mon.cm.asok mon_status
 > >> >> > [ceph@cm my-cluster]$ ^C
 > >> >> >
 > >> >> > [ceph@cm my-cluster]$ ceph-deploy gatherkeys cm
 > >> >> > [ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy
 > >> >> > gatherkeys
 > >> >> > cm
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > >> >> > /etc/ceph/ceph.client.admin. keyring
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [cm][DEBUG ] fetch remote file
 > >> >> > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > >> >> > /etc/ceph/ceph.client.admin. keyring on ['cm']
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > >> >> > /var/lib/ceph/bootstrap-osd/ ceph.keyring
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [cm][DEBUG ] fetch remote file
 > >> >> > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > >> >> > /var/lib/ceph/bootstrap-osd/ ceph.keyring on ['cm']
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > >> >> > /var/lib/ceph/bootstrap-mds/ ceph.keyring
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [cm][DEBUG ] fetch remote file
 > >> >> > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > >> >> > /var/lib/ceph/bootstrap-mds/ ceph.keyring on ['cm']
 > >> >> > [ceph@cm my-cluster]$
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > On February 14, 2014 at 8:20 AM Ashish Chandra <
 > >> >> > mail.ashishchandra@xxxxxxxxx> wrote:
 > >> >> >
 > >> >> > Hi Ron,
 > >> >> > Can you try running :
 > >> >> >
 > >> >> > ceph-deploy mon create {ceph-node}
 > >> >> >
 > >> >> > ceph-deploy gatherkeys {ceph-node}
 > >> >> >
 > >> >> >
 > >> >> > as two different commands, instead of running
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > ceph-deploy mon create-initial
 > >> >> >
 > >> >> > and let me know if the problem still persists.
 > >> >> >
 > >> >> > Actually it is not able to find "client.admin.keyring" file
as it has
 > >> >> > not
 > >> >> > been created yet.
 > >> >> >
 > >> >> > Thanks
 > >> >> >
 > >> >> > Ashish Chandra
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > On Fri, Feb 14, 2014 at 6:15 PM, Ron Gage < ron@xxxxxxxxxxx>
wrote:
 > >> >> >
 > >> >> > Greetings all!
 > >> >> > I am trying to deploy a proof of concept system for a replicated
 > >> >> > fault
 > >> >> > tolerant CIFS server cluster using CEPH as the back end. I am
having
 > >> >> > significant trouble getting the CEPH part to install correctly.
 > >> >> > Specifically, I can't get the monitor to come up into a
useful state
 > >> >> > due
 > >> >> > to
 > >> >> > the apparent lack of a keyring or two. Here is the background:
 > >> >> >
 > >> >> > Everything is running on a standalone VMware server with
internal
 > >> >> > storage.
 > >> >> > The idea here is to see it work, not to see how well it
performs.
 > >> >> >
 > >> >> > Current configuraton: 4 x OSD, 1 x monitor/manager
 > >> >> > OSD: 2 x CPU, 8 gig memory, internal only networking, 16 gig
sda for
 > >> >> > boot/root, 160 gig sdb for CEPH store, Centos 6.5 x64 fully
up to
 > >> >> > date,
 > >> >> > minimal install
 > >> >> > mon/mgr: 2 x CPU, 2 gig memory, internal and external
networking, 16
 > >> >> > gig
 > >> >> > sda, Centos 6.5 x64 fully up to date, minimal install
 > >> >> >
 > >> >> > Since I have made several attempts already to get this
working, I
 > >> >> > execute
 > >> >> > the following commands from the mgr to reset everything
(logged in as
 > >> >> > user
 > >> >> > ceph)
 > >> >> > ceph-deploy purgedata c1 c2 c3 c4 cm
 > >> >> > ceph-deploy forgetkeys
 > >> >> > ceph-deploy purge c1 c2 c3 c4 cm
 > >> >> > sudo rm -f /etc/ceph/*
 > >> >> > sudo rm -f /var/lib/ceph/mon/*.*
 > >> >> > cd ~/my-cluster
 > >> >> > rm -f *
 > >> >> >
 > >> >> > Now to deploy a straight forward single monitor node:
 > >> >> > ceph-deploy new cm
 > >> >> >
 > >> >> > And to add the public interface to the ceph.conf
 > >> >> > vi ceph.conf
 > >> >> > (add to end of global section) public network = 172.24.0.0/16
 > >> >> >
 > >> >> > Now to install the ceph software in the 5 nodes:
 > >> >> > ceph-deploy install cm c1 c2 c3 c4
 > >> >> > (waiting...waiting...waiting.. .)
 > >> >> >
 > >> >> > And here is where the problems begin - creating the initial
monitor:
 > >> >> > [ceph@cm my-cluster]$ ceph-deploy mon create-initial
 > >> >> > [ceph_deploy.cli][INFO ] Invoked (1.3.5):
/usr/bin/ceph-deploy mon
 > >> >> > create-initial
 > >> >> > [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts cm
 > >> >> > [ceph_deploy.mon][DEBUG ] detecting platform for host cm ...
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [ceph_deploy.mon][INFO ] distro info: CentOS 6.5 Final
 > >> >> > [cm][DEBUG ] determining if provided host has same hostname
in remote
 > >> >> > [cm][DEBUG ] get remote short hostname
 > >> >> > [cm][DEBUG ] deploying mon to cm
 > >> >> > [cm][DEBUG ] get remote short hostname
 > >> >> > [cm][DEBUG ] remote hostname: cm
 > >> >> > [cm][DEBUG ] write cluster configuration to
/etc/ceph/{cluster}.conf
 > >> >> > [cm][DEBUG ] create the mon path if it does not exist
 > >> >> > [cm][DEBUG ] checking for done path:
/var/lib/ceph/mon/ceph-cm/done
 > >> >> > [cm][DEBUG ] create a done file to avoid re-doing the mon
deployment
 > >> >> > [cm][DEBUG ] create the init path if it does not exist
 > >> >> > [cm][DEBUG ] locating the `service` executable...
 > >> >> > [cm][INFO ] Running command: sudo /sbin/service ceph -c
 > >> >> > /etc/ceph/ceph.conf
 > >> >> > start mon.cm
 > >> >> > [cm][DEBUG ] === mon.cm ===
 > >> >> > [cm][DEBUG ] Starting Ceph mon.cm on cm...
 > >> >> > [cm][DEBUG ] Starting ceph-create-keys on cm...
 > >> >> > [cm][WARNIN] No data was received after 7 seconds,
disconnecting...
 > >> >> > [cm][INFO ] Running command: sudo ceph --cluster=ceph
--admin-daemon
 > >> >> > /var/run/ceph/ceph-mon.cm.asok mon_status
 > >> >> > [cm][DEBUG ]
 > >> >> >
 > >> >> >
 > >> >> >
****************************** ****************************** ********************

 > >> >> > [cm][DEBUG ] status for monitor: mon.cm
 > >> >> > [cm][DEBUG ] {
 > >> >> > [cm][DEBUG ] "election_epoch": 1,
 > >> >> > [cm][DEBUG ] "extra_probe_peers": [],
 > >> >> > [cm][DEBUG ] "monmap": {
 > >> >> > [cm][DEBUG ] "created": "0.000000",
 > >> >> > [cm][DEBUG ] "epoch": 1,
 > >> >> > [cm][DEBUG ] "fsid": "b11875f0-e611-40b9-ae81- e7cba7586ed5",
 > >> >> > [cm][DEBUG ] "modified": "0.000000",
 > >> >> > [cm][DEBUG ] "mons": [
 > >> >> > [cm][DEBUG ] {
 > >> >> > [cm][DEBUG ] "addr": " 172.24.12.91:6789/0",
 > >> >> > [cm][DEBUG ] "name": "cm",
 > >> >> > [cm][DEBUG ] "rank": 0
 > >> >> > [cm][DEBUG ] }
 > >> >> > [cm][DEBUG ] ]
 > >> >> > [cm][DEBUG ] },
 > >> >> > [cm][DEBUG ] "name": "cm",
 > >> >> > [cm][DEBUG ] "outside_quorum": [],
 > >> >> > [cm][DEBUG ] "quorum": [
 > >> >> > [cm][DEBUG ] 0
 > >> >> > [cm][DEBUG ] ],
 > >> >> > [cm][DEBUG ] "rank": 0,
 > >> >> > [cm][DEBUG ] "state": "leader",
 > >> >> > [cm][DEBUG ] "sync_provider": []
 > >> >> > [cm][DEBUG ] }
 > >> >> > [cm][DEBUG ]
 > >> >> >
 > >> >> >
 > >> >> >
****************************** ****************************** ********************

 > >> >> > [cm][INFO ] monitor: mon.cm is running
 > >> >> > [cm][INFO ] Running command: sudo ceph --cluster=ceph
--admin-daemon
 > >> >> > /var/run/ceph/ceph-mon.cm.asok mon_status
 > >> >> > [ceph_deploy.mon][INFO ] processing monitor mon.cm
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][INFO ] Running command: sudo ceph --cluster=ceph
--admin-daemon
 > >> >> > /var/run/ceph/ceph-mon.cm.asok mon_status
 > >> >> > [ceph_deploy.mon][INFO ] mon.cm monitor has reached quorum!
 > >> >> > [ceph_deploy.mon][INFO ] all initial monitors are running and
have
 > >> >> > formed
 > >> >> > quorum
 > >> >> > [ceph_deploy.mon][INFO ] Running gatherkeys...
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > >> >> > /etc/ceph/ceph.client.admin. keyring
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [cm][DEBUG ] fetch remote file
 > >> >> > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > >> >> > /etc/ceph/ceph.client.admin. keyring on ['cm']
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > >> >> > /var/lib/ceph/bootstrap-osd/ ceph.keyring
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [cm][DEBUG ] fetch remote file
 > >> >> > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > >> >> > /var/lib/ceph/bootstrap-osd/ ceph.keyring on ['cm']
 > >> >> > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for
 > >> >> > /var/lib/ceph/bootstrap-mds/ ceph.keyring
 > >> >> > [cm][DEBUG ] connected to host: cm
 > >> >> > [cm][DEBUG ] detect platform information from remote host
 > >> >> > [cm][DEBUG ] detect machine type
 > >> >> > [cm][DEBUG ] fetch remote file
 > >> >> > [ceph_deploy.gatherkeys][ WARNIN] Unable to find
 > >> >> > /var/lib/ceph/bootstrap-mds/ ceph.keyring on ['cm']
 > >> >> > [ceph@cm my-cluster]$
 > >> >> >
 > >> >> >
 > >> >> > From my untrained eye, it looks like several key files aren't
either
 > >> >> > being
 > >> >> > created or copied to the expected place.
 > >> >> >
 > >> >> > Contents of ~/my-cluster:
 > >> >> > [ceph@cm my-cluster]$ ls
 > >> >> > ceph.conf ceph.log ceph.mon.keyring epel-release-6-8.noarch.rpm
 > >> >> >
 > >> >> > Contents of /var/lib/ceph/mon/ceph-cm:
 > >> >> >
 > >> >> > [ceph@cm ceph-cm]$ ls -R .: done keyring store.db sysvinit
 > >> >> >
 > >> >> > ./store.db: 000016.sst 000021.sst CURRENT LOG 000018.sst
000022.log
 > >> >> > LOCK MANIFEST-000020
 > >> >> >
 > >> >> > Contents of /etc/ceph:
 > >> >> >
 > >> >> > [ceph@cm ceph]$ ls
 > >> >> > ceph.client.admin.keyring. 5704.tmp ceph.conf rbdmap
 > >> >> >
 > >> >> > As you can plainly see, I have followed the directions in the
"quick
 > >> >> > start"
 > >> >> > deployment guide on ceph.com
 > >> >> >
 > >> >> >
( http://ceph.com/docs/master/start/quick-ceph-deploy/#create-a-cluster).
 > >> >> >
 > >> >> > What am I missing? Are the directions incomplete (or wrong)?
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > HELP!
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > Ron Gage
 > >> >> > Westland, Michigan
 > >> >> >
 > >> >> > Please be sure to check out My Photography Website
 > >> >> >
 > >> >> > ______________________________ _________________
 > >> >> > ceph-users mailing list
 > >> >> > ceph-users@xxxxxxxxxxxxxx
 > >> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > Ron Gage
 > >> >> > Westland, Michigan
 > >> >> >
 > >> >> > Please be sure to check out My Photography Website
 > >> >> >
 > >> >> > ______________________________ _________________
 > >> >> > ceph-users mailing list
 > >> >> > ceph-users@xxxxxxxxxxxxxx
 > >> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 > >> >> >
 > >> >> >
 > >> >> >
 > >> >> > Ron Gage
 > >> >> > Westland, Michigan
 > >> >> >
 > >> >> > Please be sure to check out My Photography Website
 > >> >> >
 > >> >> >
 > >> >> > ______________________________ _________________
 > >> >> > ceph-users mailing list
 > >> >> > ceph-users@xxxxxxxxxxxxxx
 > >> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 > >> >> >
 > >> > Ron Gage
 > >> > Westland, Michigan
 > >> >
 > >> > Please be sure to check out My Photography Website
 > >> >
 > > Ron Gage
 > > Westland, Michigan
 > >
 > > Please be sure to check out My Photography Website
 > >
Ron Gage
Westland, Michigan

Please be sure to check out My Photography Website

< http://www.prgstudios.com>

______________________________ _________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

--
Joao Eduardo Luis
Software Engineer | http://inktank.com | http://ceph.com

______________________________ _________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

 

Ron Gage
Westland, Michigan

Please be sure to check out My Photography Website

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com