It's possible that someone just changed the unit.meta file for all
OSDs after creation so all of them would show the same service_name
(osd.osd_spec). Are all your OSD hosts equipped with the same
hardware? So rotational and non-rotational devices? If that is the
case, you could just overwrite the host_pattern in the existing spec
to apply to all OSD nodes, apply the spec, zap and wipe the faulty OSD
and let cephadm recreate it automatically:
- ceph orch ls osd --export > osd-spec.yaml
- adapt host_pattern in osd-spec.yaml (or use labels, hosts or however
you need to distinguish between OSD and non-OSD nodes), save the file.
- ceph orch apply -i osd-spec.yaml --dry-run (just to check for any
parsing errors)
- ceph orch apply -i osd-spec.yaml (this won't do anything to existing
OSDs, only deploy new ones if there are available devices matching the
specs)
- zap & wipe your faulty OSD
- watch OSD creation
If you're unsure what cephadm will actually do (will it place the DB
on the rotational device?), you could set osd_crush_initial_weight to
0 so the OSD isn't immediately populated:
ceph config set osd osd_crush_initial_weight 0
If your OSD is created successfully and with its DB on SSD, you can
reweight the OSD (this will trigger data movement):
ceph osd crush reweight osd.<OSD_ID> <CRUSH_WEIGHT>
and then remove the osd_crush_initial_weight config:
ceph config rm osd osd_crush_initial_weight
Zitat von Gustavo Garcia Rondina <grondina@xxxxxxxxxxxx>:
From: Robert Sander <r.sander@xxxxxxxxxxxxxxxxxxx>
Sent: Friday, March 7, 2025 9:11 AM
# ceph orch ls osd --export
service_type: osd
service_id: osd_spec
service_name: osd.osd_spec
placement:
host_pattern: node-osd5
spec:
data_devices:
rotational: 1
db_devices:
rotational: 0
filter_logic: AND
objectstore: bluestore
This is just for one host.
ceph orch device zap node-osd1 /dev/sdah
And this is a different host.
That's the only one we seem to have running:
[root@node-admin1 ~]# ceph orch ls | grep osd
osd.osd_spec 167 10m ago 18M cds3-osd5
[root@node-admin1 ~]# ceph orch ls --service-type=osd -f yaml
service_type: osd
service_id: osd_spec
service_name: osd.osd_spec
placement:
host_pattern: node-osd5
spec:
data_devices:
rotational: 1
db_devices:
rotational: 0
filter_logic: AND
objectstore: bluestore
status:
created: '2023-08-23T10:43:40.474777Z'
last_refresh: '2025-03-07T15:11:17.457725Z'
running: 167
size: 167
The "running" and "size" lines shows all OSDs that we have,
could it be that this service is doing all the OSD deployments?
Thank you
-
Gustavo
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
