On 07/04/2017 06:57 AM, Z Will wrote:
Hi: I am testing ceph-mon brain split . I have read the code . If I understand it right , I know it won't be brain split. But I think there is still another problem. My ceph version is 0.94.10. And here is my test detail : 3 ceph-mons , there ranks are 0, 1, 2 respectively.I stop the rank 1 mon , and use iptables to block the communication between mon 0 and mon 1. When the cluster is stable, start mon.1 . I found the 3 monitors will all can not work well. They are all trying to call new leader election . This means the cluster can't work anymore. Here is my analysis. Because mon will always respond to leader election message, so , in my test, communication between mon.0 and mon.1 is blocked , so mon.1 will always try to be leader, because it will always see mon.2, and it should win over mon.2. Mon.0 should always win over mon.2. But mon.2 will always responsd to the election message issued by mon.1, so this loop will never end. Am I right ? This should be a problem? Or is it was just designed like this , and should be handled by human ?
This is a known behaviour, quite annoying, but easily identifiable by having the same monitor constantly calling an election and usually timing out because the peon did not defer to it.
In a way, the elector algorithm does what it is intended to. Solving this corner case would be nice, but I don't think there's a good way to solve it. We may be able to presume a monitor is in trouble during the probe phase, to disqualify a given monitor from the election, but in the end this is a network issue that may be transient or unpredictable and there's only so much we can account for.
Dealing with it automatically would be nice, but I think, thus far, the easiest way to address this particular issue is human intervention.
-Joao _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
