Hi, in our production cluster, we have the following setup
- 10 nodes
- 3 drives / server (so far), mix of SSD and HDD (different
pools) + NVMe
- dual 10G in LACP, linked to two different switches (Cisco vPC)
- OSDs, MONs and MGRs are colocated
- A + B power feeds, 2 ATS (each receiving A+B) - ATS1 and ATS2
- 2 PDU rails, each connected to an ATS (PDU1 = ATS1, PDU2 = ATS2)
- switches have dual PSUs and are connected to both rails
- CEPH nodes - single power supply
- Odd nodes (1,3,5...) are connected to PDU1
- Even nodes (2,4,6...) are connected to PDU2
... I can provide a drawing if it helps :)
Now, the default crush map ensures that multiple copies of the same
object
won't find their way on the same host, which is fine. But I'm thinking
that in case of power failure [1] of either ATS or PDU, we'd be
losing half
the nodes in the cluster at the same time. How would I go about
tuning our
map so it took into account that, for a 3 copy replicated pool, we don't
have those stored on hosts, say, 5,7,9 ?
And, what about when using EC pools ? We currently have 5+2 SSD pools -
how would we avoid losing availability in case of a power loss where 50%
of the server are offline ?
I've gone over
https://docs.ceph.com/docs/master/rados/operations/crush-map/
but don't believe I'm at the stage where I dare make changes without
incurring a huge data migration (probably can't be avoided).
Any input appreciated.
Cheers,
Phil
[1] both power feeds lost at the same time is really hard to protect
against :)
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
