Hello list, We have a Ceph cluster with two management nodes and six data nodes. Each data node has 28 HDD disks. One disk recently failed in one of the nodes, corresponding to osd.2. To replace the disk, we took the osd.2 out, stopped it, and after a few days removed it, basically: ceph osd out osd.2 ceph osd ok-to-stop osd.2 Once Ok to stop, then: ceph orch daemon stop osd.2 Once stopped: ceph osd crush remove osd.2 ceph auth del osd.2 ceph osd rm osd.2 Then checked `ceph osd tree`, `ceph orch ps`, `ceph -s`, and all confirmed that the OSD was gone. We then proceeded to physically replace the failed drive on the node. It was /dev/sdac before, and after replacing it (hot swap) the system identified it as /dev/sdah. I zapped it on the node with `sgdisk --zap-all /dev/sdah` and, back on the management node, I could see that the disk was now showing up and marked as available with `ceph orch device ls`. Then I proceeded to add a new OSD to that disk with: `ceph orch daemon add osd node-osd1:/dev/sdah`, which then failed with: Created no osd(s) on host server-osd1; already created? Which was rather strange, because total OSDs was still showing as 1 less, no new OSD was showing up in `ceph osd tree`. What was even odder is that the disk that was showing as available in `ceph orch device ls` now shows as *not* available, and looking at lsblk's output in the node it seems that it was populated by Ceph: # lsblk /dev/sdah NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdah 66:16 0 16.4T 0 disk └─ceph--157c7c44--b519--4f6d--a54b--ecd466cf81d0-osd--block--10ada574--b99c--466d--8ad6--2c97d17d1f66 253:59 0 16.4T 0 lvm Any hints on how to proceed to get the OSD added back with this disk? Thank you for any suggestions! - Gustavo _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |