On 02/14/2013 12:24 PM, Simon Leinen wrote:
Wolfgang Hennerbichler writes:
I have a ceph cluster on 2 sites. One site has 2 mons, the other site
has 1 mon. [...]
As Martin wrote, if you lose the site with the 2 mons, the entire
cluster will become unavailable.
Here's what I've been thinking to myself could be a nice solution:
Get a third site somewhere, and move one of the currently 2 mons from
your first site to that third site. The site only needs space and
performance for one (1) VM running ceph-mon. Ideally it would be
reliable, well-connected in terms of RTT etc. - but even if it isn't,
that may not matter so much.
My reasoning is that under normal conditions, the two mons in your
"real" sites will be sufficient (quorum) to maintain consistency of the
cluster. So even if the third-site mon is somehow "asleep at the
wheel", that wouldn't necessarily have any noticeable impact on your
cluster's performance. (That's pure hypothesis, I haven't tried this or
otherwise thought this through. Please comment if you disagree!)
This could actually cause problems to the quorum. If one of the
monitors doesn't perform well enough, or if it gets overloaded, you may
start seeing it behaving weirdly (constantly drop out of quorum for
instance), and that might even affect the other two monitors (dropping
out of quorum usually leads to a subsequent attempt to join it, and that
will trigger an election).
The biggest problem is still regarding latency. Given that the Paxos
will take into consideration all the monitors in the quorum for each
round, if the RTT on one of the monitors is high enough, the Paxos will
start to timeout. Timeouts make a monitor bootstrap, new election, etc.
If one of the monitors have a chronically high latency, then it may
render the cluster unusable.
It is however possible to adjust those timeouts, but have never tried to
do it as to deal with such a scenario. It ought to do it, but am not
sure what could be the potential consequences (if any) of doing so.
I guess you can hardwire the ranks of the mons to make sure that the
third-site monitor never becomes elected as leader.
ranks are assigned on ip:port. If you have some freedom when assigning
ips, then it should be fairly straightforward (127.0.0.1:6789 <
127.0.0.2:6789 < 127.0.0.2:6790)
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
