Zakhar; When adding nodes I usually set the following: noin (OSDs register as up, but stay out) norebalance (new placement shouldn't be calculated when the cluster layout changes, I've been bit by this not working as expected, so I also set below) nobackfill (PGs don't move) I then remove noin, and wait until all OSDs show as in. Then I remove norebalance, which will cause a bunch of PGs to become misplaced, as it recalculates PG locations. Finally remove nobackfill, and allow the cluster to recover completely. The first two reach steady state very quickly. Thank you, Dominic L. Hilsbos, MBA Vice President - Information Technology Perform Air International Inc. DHilsbos@xxxxxxxxxxxxxx www.PerformAir.com -----Original Message----- From: Zakhar Kirpichenko [mailto:zakhar@xxxxxxxxx] Sent: Monday, November 1, 2021 11:21 PM To: ceph-users Subject: Best way to add multiple nodes to a cluster? Hi! I have a 3-node 16.2.6 cluster with 33 OSDs, and plan to add another 3 nodes of the same configuration to it. What is the best way to add the new nodes and OSDs so that I can avoid a massive rebalance and performance hit until all new nodes and OSDs are in place and operational? I would very much appreciate any advice. Best regards, Zakhar _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |