"monclient: hunting for new mon" happens whenever the monmap changes. It
will hang if there's no quorum.
I haven't done this manually in a long time, so I'll refer to the Chef
recipes. The recipe doesn't do the 'ceph-mon add', it just starts the
daemon up.
Try:
sudo ceph-mon -i gail --mkfs --monmap /var/tmp/monmap --keyring
/var/tmp/ceph.mon.keyring
sudo ceph-mon -i gail --public-addr 172.16.1.12
sudo ceph --admin-daemon /var/run/ceph/ceph-mon.gail.asok
add_bootstrap_peer_hint 172.16.1.11
On Mon, Sep 1, 2014 at 8:21 AM, Pascal GREGIS <pgs at synerway.com> wrote:
> Hello,
>
> I am currently testing ceph to make a replicated block device for a
> project that would involve 2 data servers accessing this block device, so
> that if one fails or crashes, the data can still be used and the cluster
> can be rebuilt.
>
> This project requires that both machines run an OSD and a monitor, and
> that a 3rd monitor is run somewhere else, so that there is not a single
> point of failure.
> I know it is not the best thing to run an OSD and a monitor on the same
> machine, but I cannot really find a better solution.
>
> My problem is that, after having read several times and followed the
> documentation, I cannot succeed to add a second monitor.
>
> I have bootstrapped a first monitor, added 2 OSDs (one on the machine with
> the monitor, one on the other), and I try to add a second monitor but it
> doesn't work.
> I think I misunderstood something.
>
> Here's what I did :
>
> On the first machine named grenier:
> # setup the configuration file /etc/ceph/ceph.conf (see content further)
> # bootstrap monitor:
> $ ceph-authtool --create-keyring /var/tmp/ceph.mon.keyring --gen-key -n
> mon. --cap mon 'allow *'
> $ sudo ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring
> --gen-key -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow
> *' --cap mds 'allow'
> $ sudo chown myuser /etc/ceph/ceph.client.admin.keyring
> $ ceph-authtool /var/tmp/ceph.mon.keyring --import-keyring
> /etc/ceph/ceph.client.admin.keyring
> $ monmaptool --create --add grenier 172.16.1.11 --fsid $monuuid $tmp/monmap
> $ sudo mkdir -p /var/lib/ceph/mon/ceph-grenier
> $ sudo chown $ID -R /var/lib/ceph/mon/ceph-grenier
> $ ceph-mon --mkfs -i grenier --monmap /var/tmp/monmap --keyring
> /var/tmp/ceph.mon.keyring
> # start monitor:
> $ sudo start ceph-mon id=grenier
> # add OSD:
> $ sudo ceph osd create $osduuid
> $ sudo mkdir -p /var/lib/ceph/osd/ceph-0
> $ sudo ceph-osd -i 0 --mkfs --mkkey --osd-uuid $osduuid
> $ sudo ceph auth add osd.0 osd 'allow *' mon 'allow profile osd' -i
> /var/lib/ceph/osd/ceph-0/keyring
> $ ceph osd crush add-bucket grenier host
> $ ceph osd crush move grenier root=default
> $ ceph osd crush add osd.0 1.0 host=grenier
> # start this OSD
> $ sudo ceph-osd -i 0
>
> # copy /etc/ceph/ceph.conf, /etc/ceph/ceph.client.admin.keyring,
> /var/tmp/ceph/ceph.mon.keyring and /var/tmp/ceph/monmap from grenier to
> second node named gail:
> # add and start OSD on the second node
> $ sudo ceph osd create $newosduuid
> $ sudo mkdir -p /var/lib/ceph/osd/ceph-1
> $ sudo ceph-osd -i 1 --mkfs --mkkey --osd-uuid $newosduuid
> $ sudo ceph auth add osd.1 osd 'allow *' mon 'allow profile osd' -i
> /var/lib/ceph/osd/ceph-1/keyring
> $ ceph osd crush add-bucket gail host
> $ ceph osd crush move gail root=default
> $ ceph osd crush add osd.1 1.0 host=gail
> # start this OSD
> $ sudo ceph-osd -i 1
>
> There, everything works correctly, I can create and map a block device,
> and then write on it and the data is replicated on both nodes.
> When I perform a ceph -s I get :
> cluster a98faf65-b105-4ec7-913c-f8a33a4db4d1
> health HEALTH_OK
> monmap e1: 1 mons at {grenier=172.16.1.11:6789/0}, election epoch 2,
> quorum 0 grenier
> osdmap e13: 2 osds: 2 up, 2 in
> pgmap v47: 192 pgs, 3 pools, 0 bytes data, 0 objects
> 18400 MB used,
> 105 GB / 129 GB avail 192 active+clean
>
> And here is what I do when trying to add a second monitor on gail:
> $ sudo mkdir -p /var/lib/ceph/mon/ceph-gail
> $ ceph mon getmap -o /var/tmp/monmap
> $ sudo ceph-mon -i gail --mkfs --monmap /var/tmp/monmap --keyring
> /var/tmp/ceph.mon.keyring
> which prints:
> ceph-mon: set fsid to a98faf65-b105-4ec7-913c-f8a33a4db4d1
> ceph-mon: created monfs at /var/lib/ceph/mon/ceph-gail for mon.gail
> which seems correct (same uuid as in ceph.conf)
> $ sudo ceph-mon add gail 172.16.1.12
> This command prints:
> 2014-09-01 17:07:26.033688 7f5538ada700 0 monclient: hunting for new mon
> and hangs
>
> Then I would like to do this:
> $ sudo ceph-mon -i gail --public-addr 172.16.1.12
> but it is useless as the previous command failed.
>
>
> Would anybody guess what I am doing wrong ?
>
> I use ceph 0.80 on an Ubuntu trusty.
> My ceph.conf is as follows :
> [global]
> fsid = a98faf65-b105-4ec7-913c-f8a33a4db4d1
> mon initial members = grenier
> mon host = 172.16.1.11
> public network = 172.16.0.0/16
> auth cluster required = none
> auth service required = none
> auth client required = none
> osd journal size = 1024
> filestore xattr use omap = true
> osd pool default size = 2
> osd pool default min size = 1
> osd pool default pg num = 333
> osd pool default pgp num = 333
> osd crush chooseleaf type = 1
> _______________________________________________
> ceph-users mailing list
> ceph-users at lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140903/abc31d4b/attachment.htm>
