Re: Error on adding Monitors

["LaSalle, Jurvis" <Jurvis.Lasalle@xxxxxxxxxxxxxxxxxxxxx>]

Hi Rzk,

Thanks for the links!  I was able to add a public_network line to the config on the admin host and push the config to the nodes with a "ceph-deploy --overwrite-conf config push rc-ceph-node1 rc-ceph-node2 rc-ceph-node3".  

The bug tracker indicates that the quickstart documentation was updated four months ago to note the need for a public_network statement <http://tracker.ceph.com/issues/5195#note-15>, but I can't find the string public_network in the quickstart.  What am I missing?

Thanks,

JL

From: akatsuki <konoha.sharingan@xxxxxxxxx>
Date: Thursday, October 24, 2013 10:35 PM
To: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
Subject: Re: [ceph-users] Error on adding Monitors

Hi ,

I also found similar problem when adding a new monitor to my cluster. hope this can help.

In my ceph monitor log, i found this line:

2013-10-23 17:00:15.907105 7fc9edc9b780  0 ceph version 0.61.4 (1669132fcfc27d0c0b5e5bb93ade59d147e23404), process ceph-mon, pid 4312
2013-10-23 17:00:16.158621 7fc9edc9b780  0 mon.2 does not exist in monmap, will attempt to join an existing cluster

My server also have two NIC that use for public and private(cluster only).
and based on
http://tracker.ceph.com/issues/5195
http://cephnotes.ksperis.com/blog/2013/08/29/mon-failed-to-start

after done all step in "adding monitor" using ceph documentation,
i do this following :

Root# ceph mon dump
dumped monmap epoch 12
epoch 12
fsid b3ecd9c5-182b-4978-9272-d4b278454500
last_changed 2013-10-23 17:57:44.185915
created 2013-05-16 16:46:00.572157
0: 10.xxx.xxx.xx1:6789/0 mon.0
1: 10.xxx.xxx.xx2:6789/0 mon.1
2: 10.xxx.xxx.xx3:6789/0 mon.2

Root# ceph mon getmap -o /tmp/monmap
got latest monmap

Root#  ceph-mon -i 2 --inject-monmap /tmp/monmap

Root# /etc/init.d/ceph start mon.2
=== mon.2 ===
Starting Ceph mon.2 on ubuntuGPT3...


Regards,
Rzk

On Fri, Oct 25, 2013 at 4:46 AM, LaSalle, Jurvis <Jurvis.Lasalle@xxxxxxxxxxxxxxxxxxxxx> wrote:

I've rebuilt my nodes on raring and now I'm hitting the same issue trying
to add the 2nd and 3rd monitors as specified in the quickstart.  The
quickstart makes no mention of setting public_addr or public_network to
complete this step.  What's the deal?

JL

On 13/10/24 10:23 AM, "Joao Eduardo Luis" <joao.luis@xxxxxxxxxxx> wrote:

>On 10/24/2013 03:12 PM, David J F Carradice wrote:
>> Hi.
>>
>> I am getting an error on adding monitors to my cluster.
>> ceph@ceph-deploy:~/my-cluster$ ceph-deploy mon create ceph-osd01
>> [ceph_deploy.cli][INFO  ] Invoked (1.2.7): /usr/bin/ceph-deploy mon
>> create ceph-osd01
>> [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph-osd01
>> [ceph_deploy.mon][DEBUG ] detecting platform for host ceph-osd01 ...
>> [ceph_deploy.sudo_pushy][DEBUG ] will use a remote connection with sudo
>> [ceph_deploy.mon][INFO  ] distro info: Ubuntu 13.04 raring
>> [ceph-osd01][DEBUG ] determining if provided host has same hostname in
>> remote
>> [ceph-osd01][DEBUG ] deploying mon to ceph-osd01
>> [ceph-osd01][DEBUG ] remote hostname: ceph-osd01
>> [ceph-osd01][INFO  ] write cluster configuration to
>>/etc/ceph/{cluster}.conf
>> [ceph-osd01][DEBUG ] checking for done path:
>> /var/lib/ceph/mon/ceph-ceph-osd01/done
>> [ceph-osd01][INFO  ] create a done file to avoid re-doing the mon
>>deployment
>> [ceph-osd01][INFO  ] create the init path if it does not exist
>> [ceph-osd01][INFO  ] locating `service` executable...
>> [ceph-osd01][INFO  ] found `service` executable: /usr/sbin/service
>> [ceph-osd01][INFO  ] Running command: sudo initctl emit ceph-mon
>> cluster=ceph id=ceph-osd01
>> [ceph-osd01][INFO  ] Running command: sudo ceph --admin-daemon
>> /var/run/ceph/ceph-mon.ceph-osd01.asok mon_status
>> [ceph-osd01][ERROR ] admin_socket: exception getting command
>> descriptions: [Errno 2] No such file or directory
>> [ceph-osd01][WARNIN] monitor: mon.ceph-osd01, might not be running yet
>> [ceph-osd01][INFO  ] Running command: sudo ceph --admin-daemon
>> /var/run/ceph/ceph-mon.ceph-osd01.asok mon_status
>> [ceph-osd01][ERROR ] admin_socket: exception getting command
>> descriptions: [Errno 2] No such file or directory
>> [ceph-osd01][WARNIN] ceph-osd01 is not defined in `mon initial members`
>> [ceph-osd01][WARNIN] monitor ceph-osd01 does not exist in monmap
>> [ceph-osd01][WARNIN] neither `public_addr` nor `public_network` keys are
>> defined for monitors
>> [ceph-osd01][WARNIN] monitors may not be able to form quorum
>>
>> This is happening after a successful and first add of a monitor,
>> ceph-mon01. As per the ceph-deploy documentation, I added a single
>> monitor, then some disk daemons located on ceph-osd01~03, then went to
>> add more monitors, ceph-osd01 & 02 for a quorum. This is where I get the
>> issue.
>>
>> Is the issue related to the WARNING present regarding keys?
>
>That's a warning regarding config options (public_addr/public_network)
>and the lack of enough info to generate a monmap.
>
>>
>> It appears that when running the ceph-deploy mon create <node> from my
>> ceph-deploy server, it complains about there not being any
>> ceph-mon.<node>.asok (which I assume are address sockets). I looked in
>> the respective directories on the potential monitor nodes (which are
>> currently also the OSD nodes) and see that there is only an OSD.asok, no
>> MON.asok
>
>The monitor's .asok (admin socket) will only be created at start.  If
>the monitor hasn't been run yet, then there's no asok.
>
>>
>> I can send my ceph.conf and a brief overview if it helps.
>
>ceph.conf, specially the [global] and [mon]/[mon.foo] sections would be
>helpful.
>
>>
>> Now to add to the fun, the  ceph he alt passes, and I am able to place
>> an object. It is the fact that I cannot add more than the current single
>> mon node that I would like to resolve.
>>
>> Regards
>>
>> David
>>
>> PS Extract of ceph.log on my ceph-deploy server (separate from other

>> nodesŠ not a mon nor an osd)

>>
>> SUCCESSFUL INITIAL MON
>>
>>   764 2013-10-22 12:31:11,157 [ceph-mon01][INFO  ] Running command: sudo
>> ceph --admin-daemon /var/run/ceph/ceph-mon.ceph-mon01.asok m    764
>> on_status
>>
>>      765 2013-10-22 12:31:11,223 [ceph-mon01][DEBUG ]
>>
>>*************************************************************************
>>*******
>>
>>      766 2013-10-22 12:31:11,223 [ceph-mon01][DEBUG ] status for
>> monitor: mon.ceph-mon01
>>
>>      767 2013-10-22 12:31:11,224 [ceph-mon01][DEBUG ] {
>>
>>      768 2013-10-22 12:31:11,224 [ceph-mon01][DEBUG ]
>> ³election_epoch": 2,
>>
>>
>> FAILED ADD MON
>>
>> See above in blue.
>>
>>
>>
>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
>
>--
>Joao Eduardo Luis
>Software Engineer | http://inktank.com | http://ceph.com
>_______________________________________________
>ceph-users mailing list
>ceph-users@xxxxxxxxxxxxxx
>http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com