Re: Error on adding Monitors (Joao Eduardo Luis)

David J F Carradice <david@xxxxxxxxxxxxx> · Fri, 25 Oct 2013 09:45:24 +0100

My apologies Jaoa

I only attached my ceph.log instead of my conf file. Please find the global conf file, as generated by ceph-deploy. 

David

From:  David J F Carradice <david@xxxxxxxxxxxxx>
Date:  Friday, 25 October 2013 09:39
To:  <ceph-users@xxxxxxxxxxxxxx>
Subject:  Re: Error on adding Monitors (Joao Eduardo Luis)

Hi Joao

Please find a copy of my ceph.log from my ceph-deploy server attached.

With regards to your request for the mon.foo, could you enlighten me as to this request. I have used the ceph-deploy and am currently trying to read all of the ‘manual’ installation notes, so that I may better understand how this all works.

Regards

David

From:  <ceph-users-request@xxxxxxxxxxxxxx>
Reply-To:  <ceph-users@xxxxxxxxxxxxxx>
Date:  Thursday, 24 October 2013 15:48
To:  <ceph-users@xxxxxxxxxxxxxx>
Subject:  ceph-users Digest, Vol 9, Issue 118

Send ceph-users mailing list submissions to
	ceph-users@xxxxxxxxxxxxxx

To subscribe or unsubscribe via the World Wide Web, visit
	http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
or, via email, send a message with subject or body 'help' to
	ceph-users-request@xxxxxxxxxxxxxx

You can reach the person managing the list at
	ceph-users-owner@xxxxxxxxxxxxxx

When replying, please edit your Subject line so it is more specific
than "Re: Contents of ceph-users digest..."

Today's Topics:

   1. Re: Non-Ceph cluster name (Sage Weil)
   2. Re: Error on adding Monitors (Joao Eduardo Luis)
   3. Re: About use same SSD for OS and Journal (Kurt Bauer)
   4. Re: Hardware: SFP+ or 10GBase-T (Mark Nelson)

----------------------------------------------------------------------

------------------------------

Message: 2
Date: Thu, 24 Oct 2013 15:23:02 +0100
From: Joao Eduardo Luis <joao.luis@xxxxxxxxxxx>
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re:  Error on adding Monitors
Message-ID: <52692D46.10505@xxxxxxxxxxx>
Content-Type: text/plain; charset=windows-1252; format=flowed

On 10/24/2013 03:12 PM, David J F Carradice wrote:
 Hi.

 I am getting an error on adding monitors to my cluster.
 ceph@ceph-deploy:~/my-cluster$ ceph-deploy mon create ceph-osd01
 [ceph_deploy.cli][INFO  ] Invoked (1.2.7): /usr/bin/ceph-deploy mon
 create ceph-osd01
 [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph-osd01
 [ceph_deploy.mon][DEBUG ] detecting platform for host ceph-osd01 ...
 [ceph_deploy.sudo_pushy][DEBUG ] will use a remote connection with sudo
 [ceph_deploy.mon][INFO  ] distro info: Ubuntu 13.04 raring
 [ceph-osd01][DEBUG ] determining if provided host has same hostname in
 remote
 [ceph-osd01][DEBUG ] deploying mon to ceph-osd01
 [ceph-osd01][DEBUG ] remote hostname: ceph-osd01
 [ceph-osd01][INFO  ] write cluster configuration to /etc/ceph/{cluster}.conf
 [ceph-osd01][DEBUG ] checking for done path:
 /var/lib/ceph/mon/ceph-ceph-osd01/done
 [ceph-osd01][INFO  ] create a done file to avoid re-doing the mon deployment
 [ceph-osd01][INFO  ] create the init path if it does not exist
 [ceph-osd01][INFO  ] locating `service` executable...
 [ceph-osd01][INFO  ] found `service` executable: /usr/sbin/service
 [ceph-osd01][INFO  ] Running command: sudo initctl emit ceph-mon
 cluster=ceph id=ceph-osd01
 [ceph-osd01][INFO  ] Running command: sudo ceph --admin-daemon
 /var/run/ceph/ceph-mon.ceph-osd01.asok mon_status
 [ceph-osd01][ERROR ] admin_socket: exception getting command
 descriptions: [Errno 2] No such file or directory
 [ceph-osd01][WARNIN] monitor: mon.ceph-osd01, might not be running yet
 [ceph-osd01][INFO  ] Running command: sudo ceph --admin-daemon
 /var/run/ceph/ceph-mon.ceph-osd01.asok mon_status
 [ceph-osd01][ERROR ] admin_socket: exception getting command
 descriptions: [Errno 2] No such file or directory
 [ceph-osd01][WARNIN] ceph-osd01 is not defined in `mon initial members`
 [ceph-osd01][WARNIN] monitor ceph-osd01 does not exist in monmap
 [ceph-osd01][WARNIN] neither `public_addr` nor `public_network` keys are
 defined for monitors
 [ceph-osd01][WARNIN] monitors may not be able to form quorum

 This is happening after a successful and first add of a monitor,
 ceph-mon01. As per the ceph-deploy documentation, I added a single
 monitor, then some disk daemons located on ceph-osd01~03, then went to
 add more monitors, ceph-osd01 & 02 for a quorum. This is where I get the
 issue.

 Is the issue related to the WARNING present regarding keys?

That's a warning regarding config options (public_addr/public_network) 
and the lack of enough info to generate a monmap.

 It appears that when running the ceph-deploy mon create <node> from my
 ceph-deploy server, it complains about there not being any
 ceph-mon.<node>.asok (which I assume are address sockets). I looked in
 the respective directories on the potential monitor nodes (which are
 currently also the OSD nodes) and see that there is only an OSD.asok, no
 MON.asok

The monitor's .asok (admin socket) will only be created at start.  If 
the monitor hasn't been run yet, then there's no asok.

 I can send my ceph.conf and a brief overview if it helps.

ceph.conf, specially the [global] and [mon]/[mon.foo] sections would be 
helpful.

 Now to add to the fun, the  ceph he alt passes, and I am able to place
 an object. It is the fact that I cannot add more than the current single
 mon node that I would like to resolve.

 Regards

 David

 PS Extract of ceph.log on my ceph-deploy server (separate from other
 nodes? not a mon nor an osd)

 SUCCESSFUL INITIAL MON

   764 2013-10-22 12:31:11,157 [ceph-mon01][INFO  ] Running command: sudo
 ceph --admin-daemon /var/run/ceph/ceph-mon.ceph-mon01.asok m    764
 on_status

      765 2013-10-22 12:31:11,223 [ceph-mon01][DEBUG ]
 ********************************************************************************

      766 2013-10-22 12:31:11,223 [ceph-mon01][DEBUG ] status for
 monitor: mon.ceph-mon01

      767 2013-10-22 12:31:11,224 [ceph-mon01][DEBUG ] {

      768 2013-10-22 12:31:11,224 [ceph-mon01][DEBUG ]
 ?election_epoch": 2,

 FAILED ADD MON

 See above in blue.

 _______________________________________________
 ceph-users mailing list
 ceph-users@xxxxxxxxxxxxxx
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

-- 
Joao Eduardo Luis
Software Engineer | http://inktank.com | http://ceph.com

------------------------------

End of ceph-users Digest, Vol 9, Issue 118
******************************************

Attachment:
ceph.conf

Description: Binary data
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com