Re: ceph orch host drain daemon type

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The label removal approach is great, but it still doesn't allow you to drain only OSDs, no other daemons. I didn't think about the other labels, that's a good point. Let's see what the devs have to say. :-)

Zitat von Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx>:

Hi Eugen,

I know, but removing other services is generally done by removing labels on hosts, isn't it?

Whatever the way, another concern would be how to deal with _no_schedule and _no_conf_keyring labels when not draining all services on the host. Would it require per service type _no_schedule_labels?
I don't know. That's where devs usually chime in. :-)

Cheers,
Frédéric

----- Le 30 Aoû 24, à 12:33, Eugen Block eblock@xxxxxx a écrit :

Hi,

What would be nice is if the drain command could take care of OSDs
by default and drain all services only when called with a
--remove-all-services flag or something similar.

but that would mean that you wouldn't be able to drain only specific
services, and OSDs would be drained either way. From my perspective,
it would suffice to have the --daemon-type flag which is already
present in 'ceph orch ps --daemon-type' command. You could either
drain a specific daemon-type or drain the entire host (can be the
default with the same behaviour as it currently works). That would
allow more control about non-osd daemons.

Zitat von Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx>:

Hi Robert,

Thanks for the suggestion. Unfortunately, removing the 'osds' label
from a host does not remove the OSDs, unlike with other labels
(mons, mgrs, mdss, nfss, crash, _admin, etc.). This is because this
kind of service is tightly bound to the host and less 'portable'
than other services, I think. This is likely the purpose of the
drain command.

What would be nice is if the drain command could take care of OSDs
by default and drain all services only when called with a
--remove-all-services flag or something similar.

Frédéric

----- Le 30 Aoû 24, à 1:07, Robert W. Eckert rob@xxxxxxxxxxxxxxx a écrit :

If you are using cephadm, couldn't the host be removed from placing
osds? On my
cluster, I labeled the hosts for each service (OSD/MON/MGR/...) and have the
services deployed by label.   I believe that if you had that, then
when a label
is removed from the host the services eventually drain.



-----Original Message-----
From: Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx>
Sent: Thursday, August 29, 2024 11:30 AM
To: Eugen Block <eblock@xxxxxx>
Cc: ceph-users <ceph-users@xxxxxxx>; dev <dev@xxxxxxx>
Subject: [ceph-users] Re: ceph orch host drain daemon type

Hello Eugen,

A month back, while playing with a lab cluster, I drained a
multi-service host
(OSDs, MGR, MON, etc.) in order to recreate all of its OSDs. During this
operation, all cephadm containers were removed as expected,
including the MGR.
As a result, I got into a situation where the orchestrator backend 'cephadm'
was missing and wouldn't load anymore. I didn't have much time to
investigate
this, so I decided to recreate the lab cluster. But I think this is due to a
bug.

I would probably have avoided this situation if I had been able to ask the
orchestrator to only drain services of type 'osd'. Makes sense.

Cheers,
Frédéric.

----- Le 27 Aoû 24, à 15:19, Eugen Block eblock@xxxxxx a écrit :

Hi,

is there anything on the road map to be able to choose a specific
daemon type to be entirely removed from a host instead of all cephadm
managed daemons? I just did a quick search in tracker and github, but
it may be "hidden" somewhere else.

I was thinking about colocated daemons on a host, for example MON,
MGR, OSDs, node-exporter, crash. That's quite common, but if I just
wanted to drain all OSDs (maybe mark them as "destroyed" in order to
replace the drives), the usual 'ceph orch host drain <host>' command
would remove all daemons. That seems unnecessary if I'm going to add
the OSDs back.

Since there are a couple of other daemon types which can be deployed
multiple times per host, e. g. MDS, RGW, it doesn't only make sense
for OSDs but for other daemons as well. And those other daemons
usually have some cryptic suffix, we wouldn't need that in order to
get rid of them, it doesn't save that much time, but it could be a
nice enhancement.

Should I create a tracker issue in the enhancement section for that?

Thanks!
Eugen
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send
an email to
>> ceph-users-leave@xxxxxxx


_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx




[Index of Archives]     [CEPH Users]     [Ceph Devel]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux