Re: ceph orch host drain daemon type

[Eugen Block <eblock@xxxxxx>]

The label removal approach is great, but it still doesn't allow you to  
drain only OSDs, no other daemons. I didn't think about the other  
labels, that's a good point. Let's see what the devs have to say. :-)

Zitat von Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx>:

> Hi Eugen,
>
> I know, but removing other services is generally done by removing  
> labels on hosts, isn't it?
>
> Whatever the way, another concern would be how to deal with  
> _no_schedule and _no_conf_keyring labels when not draining all  
> services on the host. Would it require per service type  
> _no_schedule_labels?
> I don't know. That's where devs usually chime in. :-)
>
> Cheers,
> Frédéric
>
> ----- Le 30 Aoû 24, à 12:33, Eugen Block eblock@xxxxxx a écrit :
>
> > Hi,
> >
> > > What would be nice is if the drain command could take care of OSDs
> > > by default and drain all services only when called with a
> > > --remove-all-services flag or something similar.
> >
> > but that would mean that you wouldn't be able to drain only specific
> > services, and OSDs would be drained either way. From my perspective,
> > it would suffice to have the --daemon-type flag which is already
> > present in 'ceph orch ps --daemon-type' command. You could either
> > drain a specific daemon-type or drain the entire host (can be the
> > default with the same behaviour as it currently works). That would
> > allow more control about non-osd daemons.
> >
> > Zitat von Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx>:
> >
> > > Hi Robert,
> > >
> > > Thanks for the suggestion. Unfortunately, removing the 'osds' label
> > > from a host does not remove the OSDs, unlike with other labels
> > > (mons, mgrs, mdss, nfss, crash, _admin, etc.). This is because this
> > > kind of service is tightly bound to the host and less 'portable'
> > > than other services, I think. This is likely the purpose of the
> > > drain command.
> > >
> > > What would be nice is if the drain command could take care of OSDs
> > > by default and drain all services only when called with a
> > > --remove-all-services flag or something similar.
> > >
> > > Frédéric
> > >
> > > ----- Le 30 Aoû 24, à 1:07, Robert W. Eckert rob@xxxxxxxxxxxxxxx a écrit :
> > >
> > > > If you are using cephadm, couldn't the host be removed from placing
> > > > osds? On my
> > > > cluster, I labeled the hosts for each service (OSD/MON/MGR/...)  
> > > > and have the
> > > > services deployed by label.   I believe that if you had that, then
> > > > when a label
> > > > is removed from the host the services eventually drain.
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx>
> > > > Sent: Thursday, August 29, 2024 11:30 AM
> > > > To: Eugen Block <eblock@xxxxxx>
> > > > Cc: ceph-users <ceph-users@xxxxxxx>; dev <dev@xxxxxxx>
> > > > Subject:  Re: ceph orch host drain daemon type
> > > >
> > > > Hello Eugen,
> > > >
> > > > A month back, while playing with a lab cluster, I drained a
> > > > multi-service host
> > > > (OSDs, MGR, MON, etc.) in order to recreate all of its OSDs. During this
> > > > operation, all cephadm containers were removed as expected,
> > > > including the MGR.
> > > > As a result, I got into a situation where the orchestrator  
> > > > backend 'cephadm'
> > > > was missing and wouldn't load anymore. I didn't have much time to
> > > > investigate
> > > > this, so I decided to recreate the lab cluster. But I think this  
> > > > is due to a
> > > > bug.
> > > >
> > > > I would probably have avoided this situation if I had been able to ask the
> > > > orchestrator to only drain services of type 'osd'. Makes sense.
> > > >
> > > > Cheers,
> > > > Frédéric.
> > > >
> > > > ----- Le 27 Aoû 24, à 15:19, Eugen Block eblock@xxxxxx a écrit :
> > > >
> > > > > Hi,
> > > > >
> > > > > is there anything on the road map to be able to choose a specific
> > > > > daemon type to be entirely removed from a host instead of all cephadm
> > > > > managed daemons? I just did a quick search in tracker and github, but
> > > > > it may be "hidden" somewhere else.
> > > > >
> > > > > I was thinking about colocated daemons on a host, for example MON,
> > > > > MGR, OSDs, node-exporter, crash. That's quite common, but if I just
> > > > > wanted to drain all OSDs (maybe mark them as "destroyed" in order to
> > > > > replace the drives), the usual 'ceph orch host drain <host>' command
> > > > > would remove all daemons. That seems unnecessary if I'm going to add
> > > > > the OSDs back.
> > > > >
> > > > > Since there are a couple of other daemon types which can be deployed
> > > > > multiple times per host, e. g. MDS, RGW, it doesn't only make sense
> > > > > for OSDs but for other daemons as well. And those other daemons
> > > > > usually have some cryptic suffix, we wouldn't need that in order to
> > > > > get rid of them, it doesn't save that much time, but it could be a
> > > > > nice enhancement.
> > > > >
> > > > > Should I create a tracker issue in the enhancement section for that?
> > > > >
> > > > > Thanks!
> > > > > Eugen
> > > > > _______________________________________________
> > > > > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
> > > > > email to ceph-users-leave@xxxxxxx
> > > > _______________________________________________
> > > > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send
> > > > an email to
> > >> ceph-users-leave@xxxxxxx


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx