Hi, I did something like that in the past. If you have a sufficient amount of cold data in general and you can bring the OSDs back with their original IDs, recovery was significantly faster than rebalancing. It really depends how trivial the version update per object is. In my case it could re-use thousands of clean objects per dirty object. If you are unsure its probably best to do a wipe + rebalance. What can take quite a while at the beginning is the osdmap update if they were down for such a long time. The first boot until they show up as "in" will take a while. Set norecover and norebalance until you see in the OSD log that they have the latest OSD map version. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Malte Stroem <malte.stroem@xxxxxxxxx> Sent: Wednesday, September 6, 2023 4:16 PM To: ceph-mail@xxxxxxxxxxxxxxxx; ceph-users@xxxxxxx Subject: Re: Is it possible (or meaningful) to revive old OSDs? Hi ceph-mail@xxxxxxxxxxxxxxxx, you could squeeze the OSDs back in but it does not make sense. Just clean the disks with dd for example and add them as new disks to your cluster. Best, Malte Am 04.09.23 um 09:39 schrieb ceph-mail@xxxxxxxxxxxxxxxx: > Hello, > > I have a ten node cluster with about 150 OSDs. One node went down a while back, several months. The OSDs on the node have been marked as down and out since. > > I am now in the position to return the node to the cluster, with all the OS and OSD disks. When I boot up the now working node, the OSDs do not start. > > Essentially , it seems to complain with "fail[ing]to load OSD map for [various epoch]s, got 0 bytes". > > I'm guessing the OSDs on disk maps are so old, they can't get back into the cluster? > > My questions are whether it's possible or worth it to try to squeeze these OSDs back in or to just replace them. And if I should just replace them, what's the best way? Manually remove [1] and recreate? Replace [2]? Purge in dashboard? > > [1] https://docs.ceph.com/en/quincy/rados/operations/add-or-rm-osds/#removing-osds-manual > [2] https://docs.ceph.com/en/quincy/rados/operations/add-or-rm-osds/#replacing-an-osd > > Many thanks! > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |