Re: cephadm orchestrator does not restart daemons [was: ceph orch upgrade stuck between 16.2.7 and 16.2.13]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I don't really have a good idea right now, but there was a thread [1] about ssh sessions that are not removed, maybe that could have such an impact? And if you crank up the debug level to 30, do you see anything else?

ceph config set mgr debug_mgr 30


[1] https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/thread/I452F3PWBAU47R6RXMVSLIHKS4YWCFKT/

Zitat von Robert Sander <r.sander@xxxxxxxxxxxxxxxxxxx>:

On 8/15/23 16:36, Adam King wrote:

with the log to cluster level already on debug, if you do a "ceph mgr fail" what does cephadm log to the cluster before it reports sleeping? It should at least be doing something if it's responsive at all. Also, in "ceph orch ps"  and "ceph orch device ls" are the REFRESHED columns reporting that they've refreshed the info recently (last 10 minutes for daemons, last 30 minutes for devices)?

They have been refreshed very recently.

The issue seems to be a bit larger than just the not working upgrade.

We are now not even able to restart a daemon.

When I issue the command

# ceph orch daemon restart crash.cephmon01

these two lines show up in the cephadm log but nothing else happens:

2023-08-16T10:35:41.640027+0200 mgr.cephmon01 [INF] Schedule restart daemon crash.cephmon01
2023-08-16T10:35:41.640497+0200 mgr.cephmon01 [DBG] _kick_serve_loop

The container for crash.cephmon01 does not get restarted.

It looks like the service loop does not get executed.

Can we see what jobs are in this queue and why they do not get executed?

Regards
--
Robert Sander
Heinlein Consulting GmbH
Schwedter Str. 8/9b, 10119 Berlin

https://www.heinlein-support.de

Tel: 030 / 405051-43
Fax: 030 / 405051-19

Amtsgericht Berlin-Charlottenburg - HRB 220009 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux