I'm sorry, you're right — I misread it. :( But indeed step 6 is the crucial one, which tells the existing monitors to accept the new one into the cluster. You'll need to run it with an admin client keyring that can connect to the existing cluster; that's probably the part that has gone wrong. You don't need to run it from the new monitor, so if you're having trouble getting the keys to behave I'd just run it from an existing system. :) -Greg On Tue, Oct 28, 2014 at 10:11 AM, Patrick Darley <patrick.darley@xxxxxxxxxxxxxxx> wrote: > On 2014-10-28 16:08, Gregory Farnum wrote: >> >> On Mon, Oct 27, 2014 at 11:37 AM, Patrick Darley >> <patrick.darley@xxxxxxxxxxxxxxx> wrote: >>> >>> Hi there >>> >>> Over the last week or so, I've been trying to connect a ceph monitor node >>> running on a baserock system >>> to connect to a simple 3-node ubuntu ceph cluster. >>> >>> The 3 node ubunutu cluster was created by following the documented Quick >>> installation guide using 3 VMs running ubuntu Trusty. >>> >>> After the ubuntu cluster has been deployed I would then follow the >>> directions below, which I derived from comparing the ceph-deploy debug >>> information, the ceph documentation on adding monitor nodes to an >>> existing >>> system and the ceph documentation on bootstrapping monitor nodes. >>> >>> 1. scp the /etc/ceph/* from admin node >>> 2. create the dir: mkdir /var/lib/ceph/mon/ceph-bcc08 >>> 3. generate mon keyring: sudo ceph auth get mon. -o >>> /var/lib/ceph/tmp/ceph-bcc08.mon.keyring >>> 4. generate monmap: sudo ceph mon getmap -o /var/lib/ceph/tmp/monmap > > >> Yeah, this is wrong. You're here giving the monitor its own keyring >> which it is going to expect anybody to talk to to be encrypting with. > > > If you are referring to steps 3 and 4 above, I believe these are synonymous > with > steps 3 and 4 of the documentation you recommended. The monitor keyring and > the > current monmap are retrieved from the initial monitor. they are then used in > step 5 > to prepare the monitor's data directory. > > >> The docs have a section on adding monitors which should work verbatim; >> if not it's a doc bug: >> >> >> http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#adding-monitors > > > Thanks for the recommendation. I have tried to use this procedure a couple > of times but got stuck at step number > 6 of this method. The command given hangs then times out, causing the rest > of the cluster to fail. > >> -Greg > > > > Thanks for the reply! > > Much appreciated, > > Patrick > > > > > > >>> 5. That filesystem thingy: sudo ceph-mon --cluster ceph --mkfs -i bcc08 >>> --keyring /var/lib/ceph/tmp/ceph-bcc08.mon.keyring --monmap >>> /var/lib/ceph/tmp/monmap >>> 6. Unlink keys and old monmap: rm /var/lib/ceph/tmp/* >>> 7. touch things: touch /var/lib/ceph/mon/ceph-bcc08/done and touch >>> /var/lib/ceph/mon/ceph-bcc08/sysvinit >>> 8. Then start the mon: sudo /etc/init.d/ceph start mon.bcc08 >>> >>> When I carry out these steps in the attempt to add a baserock system to >>> the >>> ubuntu cluster, the monitor node has not been added to the cluster and >>> the >>> admin socket mon_status gives the following output. >>> >>> ~ # ceph --cluster=ceph --admin-daemon >>> /var/run/ceph/ceph-mon.bcc07.asok >>> mon_status >>> { "name": "bcc07", >>> "rank": -1, >>> "state": "probing", >>> "election_epoch": 0, >>> "quorum": [], >>> "outside_quorum": [], >>> "extra_probe_peers": [], >>> "sync_provider": [], >>> "monmap": { "epoch": 0, >>> "fsid": "4460079d-42f4-4e3a-8ce3-e2a7fa2685e6", >>> "modified": "2014-10-27 12:37:25.531542", >>> "created": "2014-10-27 12:37:25.531542", >>> "mons": [ >>> { "rank": 0, >>> "name": "ucc01", >>> "addr": "192.168.122.95:6789\/0"}]}} >>> >>> >>> And the newly added monitor remains stuck in the probing state >>> indefinitely. >>> To try and resolve >>> this issue I have looked at the problems monitor troubleshooting page of >>> the >>> ceph documentation, eg. ntp sychronisation and checking network >>> connectivity >>> (to the best of my ability :-s ). >>> >>> It is also worth mentioning that I have created a 3 node ceph cluster on >>> baserock machines (1 mon, 2 osds) then successfully added monitor nodes >>> running baserock and ubuntu systems using the same 8 step process given >>> above. >>> >>> This leaves me confused as to why adding the monitor run on baserock to >>> the >>> all ubuntu cluster specifically is causing problems. >>> >>> Are there any reasons why this 'probing' problem could be occuring? Im >>> feeling a little stuck of how to proceed and would welcome any >>> suggestions. >>> >>> Thanks for your help, >>> >>> Patrick >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users@xxxxxxxxxxxxxx >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |