Howdy,
I seem to be facing a problem on my 16.2.9 ceph cluster. After a staggered reboot of my 3 infra nodes all of ceph orch commands are hanging much like in this previous reported issue [1]
I have paused orch and rebuilt a manager by hand as outlined here [2], and the issue continues to persist. I am unable to scale up or down of services, restart daemons, etc.
ceph orch ls –verbose
<snip>
[{'flags': 8,
'help': 'List services known to orchestrator',
'module': 'mgr',
'perm': 'r',
'sig': [argdesc(<class 'ceph_argparse.CephPrefix'>, req=True, name=prefix, n=1, numseen=0, prefix=orch),
argdesc(<class 'ceph_argparse.CephPrefix'>, req=True, name=prefix, n=1, numseen=0, prefix=ls),
argdesc(<class 'ceph_argparse.CephString'>, req=False, name=service_type, n=1, numseen=0),
argdesc(<class 'ceph_argparse.CephString'>, req=False, name=service_name, n=1, numseen=0),
argdesc(<class 'ceph_argparse.CephBool'>, req=False, name=export, n=1, numseen=0),
argdesc(<class 'ceph_argparse.CephChoices'>, req=False, name=format, n=1, numseen=0, strings=plain|json|json-pretty|yaml|xml-pretty|xml),
argdesc(<class 'ceph_argparse.CephBool'>, req=False, name=refresh, n=1, numseen=0)]}]
Submitting command: {'prefix': 'orch ls', 'target': ('mon-mgr', '')}
submit {"prefix": "orch ls", "target": ["mon-mgr", ""]} to mon-mgr
<hang>
Debug output on the manager:
debug 2022-07-22T23:27:12.509+0000 7fc180230700 0 log_channel(audit) log [DBG] : from='client.1084220 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch
I have collected a startup of the manager and uploaded it for review [3]
Many Thanks,
Tim
[1] https://www.spinics.net/lists/ceph-users/msg68398.html
[2] https://docs.ceph.com/en/quincy/cephadm/troubleshooting/#manually-deploying-a-mgr-daemon
[3] https://pastebin.com/Dvb8sEbz
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
