Immediate thought: Forget about crush maps, osds, etc. If you lose half
the nodes (when one power rail fails) your MONs will lose quorum. I
don't see how you can win with that configuration...
On 28/05/2020 13:18, Phil Regnauld wrote:
Hi, in our production cluster, we have the following setup
- 10 nodes
- 3 drives / server (so far), mix of SSD and HDD (different pools) + NVMe
- dual 10G in LACP, linked to two different switches (Cisco vPC)
- OSDs, MONs and MGRs are colocated
- A + B power feeds, 2 ATS (each receiving A+B) - ATS1 and ATS2
- 2 PDU rails, each connected to an ATS (PDU1 = ATS1, PDU2 = ATS2)
- switches have dual PSUs and are connected to both rails
- CEPH nodes - single power supply
- Odd nodes (1,3,5...) are connected to PDU1
- Even nodes (2,4,6...) are connected to PDU2
... I can provide a drawing if it helps :)
Now, the default crush map ensures that multiple copies of the same object
won't find their way on the same host, which is fine. But I'm thinking
that in case of power failure [1] of either ATS or PDU, we'd be losing half
the nodes in the cluster at the same time. How would I go about tuning our
map so it took into account that, for a 3 copy replicated pool, we don't
have those stored on hosts, say, 5,7,9 ?
And, what about when using EC pools ? We currently have 5+2 SSD pools -
how would we avoid losing availability in case of a power loss where 50%
of the server are offline ?
I've gone over https://docs.ceph.com/docs/master/rados/operations/crush-map/
but don't believe I'm at the stage where I dare make changes without
incurring a huge data migration (probably can't be avoided).
Any input appreciated.
Cheers,
Phil
[1] both power feeds lost at the same time is really hard to protect against :)
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
