Hi,
I have a different approach in mind for a replacement, we successfully
accomplished that last year in our production environment where we
replaced all nodes of the cluster with newer hardware. Of course we
wanted to avoid rebalancing the data multiple times.
What we did was to create a new "root" bucket in our crush tree
parallel to the root=default, then we moved the old nodes to the new
root. This can't trigger rebalancing because there was no host
available in the default root anymore, but the data was still
available to the clients as if nothing had changed.
Then we added the new nodes to the default root with initial osd crush
weight = 0. After all new nodes and OSDs were there we increased the
weight to start data movement. This way all PGs were recreated only
once on the new nodes, slowly draining the old servers.
This should be a valid approach for a single server, too. Create a
(temporary) new root or bucket within your crush tree and move the old
host to that bucket. Then add your new server to the correct root with
initial osd crush weight = 0. When all OSDs are there, increase the
weight for all OSDs at once to start the data movement.
This was all in a Luminous cluster.
Regards,
Eugen
Zitat von Nghia Viet Tran <Nghia.Viet.Tran@xxxxxxxxxx>:
Hi everyone,
I'm working on replacing OSDs node with the newer one. The new host
has the new hostname and new disk (faster one but the same size with
old disk). My plan is
- Reweight the OSD to zero to spread all existed data to the rest
nodes to keep data availability
- set flag noout, norebalance, norecover, nobackfill, destroy the
OSD and join the new OSD as the same ID of the old one.
By above approach, the cluster will remap PGs of all nodes. Each
data will be moved twice times until it reach the new OSD (reweight
and join new node as same ID)
I also did the other way that only set flags and destroy OSD. But
the result is still the same (degraded objects from destroyed osd
and misplaced object after joining new osd)
Are there any ways to replace the OSD node directly without
remapping PGs of the whole cluster?
Many thanks!
Nghia.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
