Never mind, problem solved. Cluster is now healthy again. J
Tried the recovery process again and managed to remove the hung monitor. Not sure why it didn’t work last time, but hey.
From: Karan Singh [mailto:ksingh@xxxxxx]
Sent: Wednesday, October 30, 2013 2:49 PM
To: Whittle, Alistair: Investment Bank (LDN)
Cc: ceph-users@xxxxxxxxxxxxxx
Subject: Re: Ceph monitor problems
Hi AW
Did you check on firewall ( iptables ) between the nodes. If this is a test cluster disable iptables and try.
Regards
Karan Singh
From: "alistair whittle" <alistair.whittle@xxxxxxxxxxxx>
To: ceph-users@xxxxxxxxxxxxxx
Sent: Wednesday, 30 October, 2013 4:20:27 PM
Subject: Ceph monitor problems
All,
My ceph cluster is failing to respond to calls for health status etc. It simply hangs at each command and then tells me there is an error connecting to the cluster. I assume that it is because I tried to add a second monitor which seemed to hang during the “hunting” phase. I eventually had to kill the tasks which I think resulted in my monmap becoming corrupt. I have tried to access it from the only remaining working monitor, but have not had much joy with that.
I have also tried to follow the docs re removing monitors from unhealthy clusters, but have as yet had no luck with getting the stable monmap extracted....
When I run any ceph command from the remaining monitor host I get the following error:
2013-10-30 14:19:01.822660 7f16080f2700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication
2013-10-30 14:19:01.822743 7f16080f2700 0 librados: client.admin initialization error (2) No such file or directory
Error connecting to cluster: ObjectNotFound
It’s the hunting part which implies that something is still stuck from when I tried to add the second monitor.
Any ideas on how I can fix the monmap so the remaining monitor can be used again?
Thanks.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
