Hello, your subject is misleading, as this is not really related to Deis/Docker. Find the very recent "Is mon initial members used after the first quorum?" thread in this ML. In short, list all your 3 mons in the initial members section. And yes, rebooting things all at the same time can be "fun", I managed to get into a similar situation like yours once (though that cluster really had only one mon) and it took a hard reset to fix things eventually. Christian On Tue, 16 Dec 2014 08:52:15 +0800 Jimmy Chu wrote: > Hi, > > I installed ceph on 3 nodes, having one monitor, and one OSD running on > each node. After rebooting them all at once (I see this may be a bad > move now), the ceph monitors refuse to connect to each other. > > When I run: > > ceph mon getmap -o /etc/ceph/monmap > > or even > > ceph -s > > It only shows the following: > > Dec 14 16:38:44 deis-1 sh[933]: 2014-12-14 08:38:44.265419 7f5cec71f700 > 0 -- :/1000021 >> 10.132.183.191:6789/0 pipe(0x7f5ce40296a0 sd=4 :0 s=1 > pgs=0 cs=0 l=1 > c=0x7f5ce4029930).fault > Dec 14 16:38:44 deis-1 sh[933]: 2014-12-14 08:38:44.265419 7f5cec71f700 > 0 -- :/1000021 >> 10.132.183.192:6789/0 pipe(0x7f5ce40296a0 sd=4 :0 s=1 > pgs=0 cs=0 l=1 > c=0x7f5ce4029930).fault > Dec 14 16:38:50 deis-1 sh[933]: 2014-12-14 08:38:50.267398 7f5cec71f700 > 0 -- :/1000021 >> 10.132.183.190:6789/0 pipe(0x7f5cd40030e0 sd=4 :0 s=1 > pgs=0 cs=0 l=1 > c=0x7f5cd4003370).fault > ...keep repeating... > > So, there is no quorum formed, and ceph admin socket file is not there > for connection. What should be my next step to recover the storage? > > This is *my /etc/ceph/ceph.conf file*: > [global] > fsid = cc368515-9dc6-48e2-9526-58ac4cbb3ec9 > mon initial members = deis-3 > auth cluster required = cephx > auth service required = cephx > auth client required = cephx > osd pool default size = 3 > osd pool default min_size = 1 > osd pool default pg_num = 128 > osd pool default pgp_num = 128 > osd recovery delay start = 15 > log file = /dev/stdout > > [mon.deis-3] > host = deis-3 > mon addr = 10.132.183.190:6789 > > [mon.deis-1] > host = deis-1 > mon addr = 10.132.183.191:6789 > > [mon.deis-2] > host = deis-2 > mon addr = 10.132.183.192:6789 > > [client.radosgw.gateway] > host = deis-store-gateway > keyring = /etc/ceph/ceph.client.radosgw.keyring > rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock > log file = /dev/stdout > > Thank you. > > - Jimmy Chu -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Fusion Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |