Thanks. This is one way to go, of course (I've been thinking about this). In my situation things are a little different, as I have placement rules that would store some data exclusively on the first location, and different data exclusively on the second location, because the replication speed between the sites isn't very fast (100 MBit). I've ended up having two ceph clusters. Wolfgang On 02/18/2013 02:09 PM, Sébastien Han wrote: > Hi, > > I don't like to promote my own website on the mailing but this article > tries to address your problem with a possible answer :-) > > http://www.sebastien-han.fr/blog/2013/01/28/ceph-geo-replication-sort-of/ > -- > Regards, > Sébastien Han. > > > On Thu, Feb 14, 2013 at 2:03 PM, Joao Eduardo Luis > <joao.luis@xxxxxxxxxxx> wrote: >> On 02/14/2013 12:24 PM, Simon Leinen wrote: >>> >>> Wolfgang Hennerbichler writes: >>>> >>>> I have a ceph cluster on 2 sites. One site has 2 mons, the other site >>>> has 1 mon. [...] >>> >>> >>> As Martin wrote, if you lose the site with the 2 mons, the entire >>> cluster will become unavailable. >>> >>> Here's what I've been thinking to myself could be a nice solution: >>> >>> Get a third site somewhere, and move one of the currently 2 mons from >>> your first site to that third site. The site only needs space and >>> performance for one (1) VM running ceph-mon. Ideally it would be >>> reliable, well-connected in terms of RTT etc. - but even if it isn't, >>> that may not matter so much. >>> >>> My reasoning is that under normal conditions, the two mons in your >>> "real" sites will be sufficient (quorum) to maintain consistency of the >>> cluster. So even if the third-site mon is somehow "asleep at the >>> wheel", that wouldn't necessarily have any noticeable impact on your >>> cluster's performance. (That's pure hypothesis, I haven't tried this or >>> otherwise thought this through. Please comment if you disagree!) >> >> >> This could actually cause problems to the quorum. If one of the monitors >> doesn't perform well enough, or if it gets overloaded, you may start seeing >> it behaving weirdly (constantly drop out of quorum for instance), and that >> might even affect the other two monitors (dropping out of quorum usually >> leads to a subsequent attempt to join it, and that will trigger an >> election). >> >> The biggest problem is still regarding latency. Given that the Paxos will >> take into consideration all the monitors in the quorum for each round, if >> the RTT on one of the monitors is high enough, the Paxos will start to >> timeout. Timeouts make a monitor bootstrap, new election, etc. If one of >> the monitors have a chronically high latency, then it may render the cluster >> unusable. >> >> It is however possible to adjust those timeouts, but have never tried to do >> it as to deal with such a scenario. It ought to do it, but am not sure what >> could be the potential consequences (if any) of doing so. >> >> >>> I guess you can hardwire the ranks of the mons to make sure that the >>> third-site monitor never becomes elected as leader. >> >> >> ranks are assigned on ip:port. If you have some freedom when assigning ips, >> then it should be fairly straightforward (127.0.0.1:6789 < 127.0.0.2:6789 < >> 127.0.0.2:6790) >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- DI (FH) Wolfgang Hennerbichler Software Development Unit Advanced Computing Technologies RISC Software GmbH A company of the Johannes Kepler University Linz IT-Center Softwarepark 35 4232 Hagenberg Austria Phone: +43 7236 3343 245 Fax: +43 7236 3343 250 wolfgang.hennerbichler@xxxxxxxxxxxxxxxx http://www.risc-software.at _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |