2012/1/19 Tommi Virtanen <tommi.virtanen@xxxxxxxxxxxxx>: > On Wed, Jan 18, 2012 at 12:45, Andrey Stepachev <octo47@xxxxxxxxx> wrote: >> I understand this. But I want to know, how ceph will react on >> some bad conditions. I don't want to lost my data because >> of some broken hardware (like switch) do some bad things on >> my cluster. (so far, ceph looks good event in such bad environment). > > When a node becomes unreachable / equivalently slow, Ceph clients will > wait for it. After a brief period of time, the node is declared down, > and the operations will continue -- the Placement Group(s) involved > will be in degraded mode, and you won't have all the replicas of the > data you want. Once the node recovers (goes up) or is declared lost > (goes out), replicas of the relevant data will be placed elsewhere. > Either way, once the data is at the desired replication level again, > the PG gets out of degraded mode. Thanks for clarification. I don't dig yet, how data moved when osd marked as out and how pg calculated for this data. (in other words, how new place for missing replica calculated, and what happens, when out osd returns back) > > We don't recommend running Ceph in a WAN scenario. If you care about > your not losing your data, you should still be able to build > independent failure domains just by using different rows/areas/racks > in a single data center. Low latency and high bandwidth are very > desirable. Ceph does synchronous replication instead of the more > common "slave trails the master by some unknown amount of operations". We don't try to use Ceph in true WAN. But we try to find some dfs, which will operate well in our environment: multiple dc, relatively low latency (<10ms as average), irregular dc outages. And we need synchronous replication and we need bigtable (hbase in case of ceph). I believe, that ceph can be configured to operate in such environment, but I can be completely wrong. And I try to check some boundary conditions. > >>> Object removal is done in the background. Give it some time, and good >>> networking, and you should get the free space back. >> So, I can't 'trigger' this procedure? >> (I can't find good docs about scrub and repair, so I still don't >> know what they do exactly). > > It's triggered automatically, just give it idle time and it should get it done. Yeah, it is good to see some progress (like mdadm shows), but it is not critical. Can you point me, who does this job. MDS? -- Andrey. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
