I am aware of monitor consensus requirement. It is taken care of (there is a third room with only monitor node). My problem is about OSD redundancy, since I can only use 2 server rooms for OSDs. I could use EC-pools, lrc or any other ceph configuration. But I could not find a configuration that would address the issue. The write acknowledge rule should read something like this: 1. If both rooms are "up", do not acknowledge write until ack is received from both rooms. 2. If only one room is "up" (forget rule 1.) acknowledge write on the first ack. The ceph documentation talks about recursively defined locality sets, so I assume it allows for different rules on room/rack/host... levels. But as far as I can see, it can not depend on "room" availability. Is this possible to configure? I would appreciate example configuration commands. regards, Zoran ________________________________________ From: Eino Tuominen <eino@xxxxxx> Sent: Wednesday, February 28, 2018 8:47 AM To: Zoran Bošnjak; ceph-users@xxxxxxxx Subject: Re: mirror OSD configuration > Is it possible to configure crush map such that it will tolerate "room" failure? In my case, there is one > network switch per room and one power supply per room, which makes a single point of (room) failure. Hi, You cannot achieve real room redundancy with just two rooms. At minimum you'll need a third room (witness) from which you'll need independent network connections to the two server rooms. Otherwise it's impossible to have monitor quorum when one of the two rooms fails. And then you'd need to consider osd redundancy. You could do with replica size = 4, min_size = 2 (or any min_size = n, size = 2*n ), but that's not perfect as you lose exactly half of the replicas in case of a room failure. If you were able to use EC-pools you'd have more options with LRC coding (http://docs.ceph.com/docs/master/rados/operations/erasure-code-lrc/). We run ceph in a 3 room configuration with 3 monitors, size=3, min_size=2. It works, but it's not without hassle either. -- Eino Tuominen _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |