|
Hi all,
We're trying to spread data in our ceph cluster as much as possible,
that is pick different racks, then different hosts, then different OSDs.
It seems to work fine as long as there are enough buckets available,
but if we ask for more replicas than we have racks, for instance, the
requested number of replicas is not achieved:
For example, what we've seen is that with a replication size of 4, a rule like
rule metadata {
ruleset 1 type replicated min_size 1 max_size 10 step take default step chooseleaf firstn 0 type rack step emit } and only 3 (!) racks we get only 3 replicas, like
osdmap e3078 pg 1.ba (1.ba) -> up [116,37,161] acting [116,37,161]
What we'd like is basically that crush tries to find 4 different racks (for the 4
replicas), and if it finds only 3, pick 4 different hosts across the 3 racks.
Is there an easy way to do this?
BTW, health is OK despite having not enough replicas for the pool. What's
the best way to detect such situations where the actual state deviates from
the desired state?
TIA,
Arne
--
Arne Wiebalck
CERN IT
|
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
