----- Le 25 Oct 24, à 18:21, Frédéric Nass frederic.nass@xxxxxxxxxxxxxxxx a écrit :
> ----- Le 25 Oct 24, à 16:31, Bob Gibson rjg@xxxxxxxxxx a écrit :
>
>> HI Frédéric,
>>
>>> I think this message shows up as this very specific post adoption 'osd' service
>>> has already been marked as 'deleted'. Maybe when you ran the command for the
>>> first time.
>>> The only reason it still shows up on 'ceph orch ls' is that 95 OSDs are still
>>> referencing this service in their configuration.
>>>
>>> Once you'll have edited all OSDs /var/lib/ceph/$(ceph fsid)/osd.xxx/unit.meta
>>> files (changed their service_name) and restarted all OSDs (or recreated these
>>> 95 OSDs encrypted under another service_name), the 'osd' service will disappear
>>> by itself and won't show up anymore on 'ceph orch ls' output. At least this is
>>> what I've observed in the past.
>>
>> Yes, as Eugen pointed out, it doesn’t make sense to try to delete an unmanaged
>> service using the orchestrator.
>
> Well actually you **can** delete a service whatever it's status (managed or
> unmanaged).
To explain a bit more, see below:
$ ceph orch ls --export osd osd.delete
service_type: osd
service_id: delete
service_name: osd.delete
placement:
hosts:
- test-mom02h01
unmanaged: true
spec:
data_devices:
size: :11G
db_devices:
size: '12G:'
db_slots: 2
filter_logic: AND
objectstore: bluestore
This is what you should expect:
$ ceph orch rm osd.delete
Error EINVAL: If osd.delete is removed then the following OSDs will remain, --force to proceed anyway
host test-mom02h01: osd.11
$ ceph orch rm osd.delete --force <--- ok, let's force it
Removed service osd.delete
$ ceph orch ls | grep osd
osd.delete 1 95s ago - <unmanaged> <--- still here because used by 1 OSD
osd.standard 12 9m ago 8w label:osds
$ ceph orch rm osd.delete
Invalid service 'osd.delete'. Use 'ceph orch ls' to list available services. <--- but not for the orchestrator
$ sed -i 's/osd.delete/osd.standard/g' /var/lib/ceph/$(ceph fsid)/osd.11/unit.meta <--- remove this service from osd.11 configuration
$ ceph orch daemon restart osd.11
Scheduled to restart osd.11 on host 'test-mom02h01'
$ ceph orch ls | grep osd
osd.standard 13 8m ago 8w label:osds <--- osd.delete service finally gone
The osd.delete service is finally gone right after the last OSD stopped referencing it.
With the very specific post adoption 'osd' service, when you try to delete it, it doesn't complain about existing OSDs referencing it (when it should...) and doesn't require you to use the --force argument.
It just deletes the service (that will finally be removed when no more OSDs are using it).
The fact that the 'ceph orch rm' output is not consistent when deleting the 'osd' post adoption service and deleting any other osd service that you create looks like a bug to me.
But anyways, that was just to say you can delete an osd service whatever it's status (managed or unmanaged).
Cheers,
Frédéric.
> It works just fine with any osd service other than this specific post adoption
> 'osd' service. Don't know why.
>
> Frédéric.
>
>>
>>>> My hunch is that some persistent state is corrupted, or there’s something else
>>>> preventing the orchestrator from successfully refreshing its device status, but
>>>> I don’t know how to troubleshoot this. Any ideas?
>>>
>>> I don't think this is related to the 'osd' service. As suggested by Tobi,
>>> enabling cephadm debug will tell you more.
>>
>> Agreed. I’ll dig through the logs some more today to see if I can spot any
>> problems.
>>
>> Cheers,
>> /rjg
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
