Hello,
i seem to not have removed old osd. Now i have:
root@ceph07:/tmp# ceph orch ps |grep -e error -e stopped |grep ceph07
_osd.33 ceph07 stopped 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
mon.ceph01 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.0 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.1 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.11 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.12 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.14 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.18 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.22 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.30 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.4 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.64 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
osd.8 ceph07 error 2h ago 2y
<unknown> quay.io/ceph/ceph:v15.2.17 <unknown> <unknown>
Which are non-existing Daemons on that node "ceph07". And i can not remove them:
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.0 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.1 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.12 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.14 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.18 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.30 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.4 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.64 --force
cephadm rm-daemon --fsid=5436dd5d-83d4-4dc8-a93b-60ab5db145df
--name=osd.8 --force
root@ceph07:/tmp# ls /var/lib/ceph/5436dd5d-83d4-4dc8-a93b-60ab5db145df/
crash home osd.66 osd.67 osd.68 osd.69 osd.999 removed
=> thats correct.
root@ceph07:/tmp# ls /var/lib/ceph/5436dd5d-83d4-4dc8-a93b-60ab5db145df/removed/
mon.ceph01_2020-09-02T07:11:30.232540
mon.ceph07_2020-11-20T14:17:56.122749
osd.12_2022-12-29T13:17:47.855132 osd.22_2022-12-29T13:13:47.233379
osd.64_2022-12-29T13:17:50.732467 osd.73_2022-12-29T09:54:58.009039Z
mon.ceph01_2022-12-29T13:18:33.702553
osd.0_2022-12-29T13:17:46.661637
osd.14_2022-12-29T13:17:48.485548 osd.30_2022-12-29T13:17:49.685540
osd.70_2022-12-29T09:56:15.014346Z osd.74_2022-12-29T09:54:59.529058Z
mon.ceph02_2020-09-01T12:07:11.808391
osd.11_2022-12-29T13:15:39.944974
osd.18_2022-12-29T13:17:49.145034 osd.32_2020-07-30T09:44:23.252102
osd.71_2022-12-29T09:54:55.157744Z osd.75_2022-12-29T09:55:02.647709Z
mon.ceph03_2020-09-01T13:26:34.704724
osd.1_2022-12-29T13:17:47.233991
osd.20_2022-12-29T12:58:27.511277 osd.4_2022-12-29T13:17:50.199486
osd.72_2022-12-29T09:54:56.537846Z osd.8_2022-12-29T13:17:51.372638
my first try was to rename the old/non-active OSD from osd.33 to
_osd.33, but now i have a dangling module here:
root@ceph07:/tmp# ceph -s
cluster:
id: 5436dd5d-83d4-4dc8-a93b-60ab5db145df
health: HEALTH_ERR
mons are allowing insecure global_id reclaim
20 failed cephadm daemon(s)
Module 'cephadm' has failed: '_osd'
Any hints on how to clean up my node? :)
Cheers,
Mario
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
