In addition to the workload distribution, the number of mons will also determine how resilliant your system is towards admin mistakes. With 5 mons, you can do service on 1 and loose and additional one without loosing service. I have been there and would not operate a production cluster with less than 5 MONs. The level of redundancy decides in how deep trouble you can allow yourself to get into without loosing service. Minimum requirements only work well in nice weather. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Tyler Stachecki <stachecki.tyler@xxxxxxxxx> Sent: 27 January 2022 14:19:52 To: Zhenshi Zhou Cc: ceph-users Subject: [Warning Possible spam] Re: What exactly does the number of monitors depends on They maintain and distribute the OSD map, i.e. the state of the OSDs. Generally 3 or 5 recommended as there always needs to be a quorum of mons available or the cluster can stall. With 3 mons, you can bring down one for servicing, patching, etc. but if another mon fails while you intentionally take one down, that leaves only one mon up and it will not have quorum, and not be able to function. The mons also have to do CPU intensive work with PGs when you are changing charteristics of the pools, so for large clusters you will want enough of them such that you have enough headroom for the PG operations while still distributing maps. Look at the mon_cpu_threads tunable in the code. So for large clusters, 5 mons is advised to provide a little more buffer... you may feel that 3 is fine for smaller clusters, though. On Thu, Jan 27, 2022, 7:56 AM Zhenshi Zhou <deaderzzs@xxxxxxxxx> wrote: > Hi all, > > What makes me confused recently is that how to decide the number of > monitors in a cluster. > > In my opinion, it depends on the size of the cluster while my colleague > says it should be at least 5 in a cluster. He sends me the redhat document > < > https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/5/html-single/installation_guide/index > > > and > it is indeed recommended to have 5 monitors in a cluster(see section 3.12). > > However it also recommends deploying at least 3 monitors, deploying 5 > monitors for the cluster exceeding 750 osds. > > So what exactly does the number of monitors depends on? > > Thanks > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |