Re: help, add mon failed lead to cluster failure

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 26/03/14 10:40, duan.xufeng@xxxxxxxxxx wrote:

Hi,
         I just add a new mon to a health cluster by following website
manual "http://ceph.com/docs/master/rados/operations/add-or-rm-mons/";
"ADDING MONITORS" step by step,

but when i execute step 6:
ceph mon add <mon-id> <ip>[:<port>]

the command didn't return, then i execute "ceph -s" on health mon node,
this command didn't return either.

so i try to restart mon to recover the whole cluster, but it seems never
recover.

Please anyone tell me how to deal with it?

First you should make sure you are able to reach the first monitor from the new monitor's server. A simple 'telnet IP PORT' should be enough, with IP being the IP of the server where the monitor lives, PORT being the monitor's port (6789 by default).

If that fails, you should probably check firewall rules dropping/rejecting connections between the servers, make sure the servers are on the same subnets, etc.

You may want to check both monitor's monmaps and mon status via the admin socket: 'ceph daemon mon.FOO mon_status', with FOO being the monitor's id. This command must be run on the monitor's server.

If all else fails, please set 'debug mon = 10' and 'debug ms = 1' on both monitors, restart them and send us the logs.

My guess is that there's something preventing the new monitor from reaching the first monitor, and the first monitor is unable to form a quorum just by itself (as it needs a second monitor to join in order to establish a majority).

You may also want to check the monitor troubleshooting section on the docs page:

http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/

  -Joao


--
Joao Eduardo Luis
Software Engineer | http://inktank.com | http://ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux