Thanks for the response. I did forget to mention that NTP is setup and does appear to be running (just double checked).
Is this good enough resolution?
$ for node in $nodes; do ssh tvsa${node} sudo date --rfc-3339=ns; done
2014-12-09 09:15:39.404292557-08:00
2014-12-09 09:15:39.521762397-08:00
2014-12-09 09:15:39.641491188-08:00
2014-12-09 09:15:39.761937524-08:00
2014-12-09 09:15:39.911416676-08:00
2014-12-09 09:15:40.029777457-08:00
Bill
________________________________________
From: Rodrigo Severo [rodrigo@xxxxxxxxxxxxxxxxxxx]
Sent: Tuesday, December 09, 2014 4:02 AM
To: Sanders, Bill
Cc: ceph-users@xxxxxxxxxxxxxx
Subject: Re: Monitors repeatedly calling for new elections
On Mon, Dec 8, 2014 at 5:23 PM, Sanders, Bill <Bill.Sanders@xxxxxxxxxxxx> wrote:
> Under activity, we'll get monitors going into election cycles repeatedly,
> OSD's being "wrongly marked down", as well as slow requests "osd.11
> 39.7.48.6:6833/21938 failed (3 reports from 1 peers after 52.914693 >= grace
> 20.000000)" . During this, ceph -w shows the cluster essentially idle.
> None of the network, disks, or cpu's ever appear to max out. It also
> doesn't appear to be the same OSD's, MON's, or node causing the problem.
> Top reports all 128 GB RAM (negligible swap) in use on the storage nodes.
> Only Ceph is running on the storage nodes.
I'm really new to Ceph but my first bet is that your computers aren't
clock synchronized. Are all of them with working ntpds?
Regards,
Rodrigo Severo
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
