I found a few WAN RBD cluster design discussions, but not a local one, so was wonderinng if anyone has experience with a resilience-oriented short distance (<10 km, redundant fiber connections) cluster in two datacenters with a third site for quorum purposes only? I can see two types of scenarios: 1. Two (or even number) of OSD nodes at each site, 4x replication (size 4, min_size 2). Three MONs, one at each site to handle split brain. Question: How does the cluster handle the loss of communication between the OSD sites A and B, while both can communicate with the quorum site C? It seems, one of the sites should suspend, as OSDs will not be able to communicate between sites. 2. 3x replication for performance or cost (size 3, min_size 2 - or even min_size 1 and strict monitoring). Two replicas and two MONs at one site and one replica and one MON at the other site. Question: in case of a permanent failure of the main site (with two replicas), how to manually force the other site (with one replica and MON) to provide storage? I would think a CRUSH map change and modifying ceph.conf to include just one MON, then build two more MONs locally and add? -- Alex Gorbachev Storcium _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |