Re: Ceph Dashboard suddenly gone and primary remote is not accessible [CEPHADM_HOST_CHECK_FAILED, CEPHADM_REFRESH_FAILED]

[Gencer W. Genç <gencer@xxxxxxxxxxxxx>]

Hi again,

One more update:

I connected to server-2 and ran ceph -s there. I got:

Error initializing cluster client: ObjectNotFound('RADOS object not found (error calling conf_read_file)')


Today I created a RBD pool and created 2 RDB images in this pool. Could this be reason for all dashboard issue and this?

Thanks,
Gencer.
On 19.05.2020 23:44:25, Gencer W. Genç <gencer@xxxxxxxxxxxxx> wrote:
Hi, 

I was browsing dashboard today. Then suddently it stopped working and i got 502 errors. I checked via root login and see thet ceph health is down to WARN.

I can access all rdb devices and CephFS. They work. All OSDs in server-1 is up.


    health: HEALTH_WARN
            1 hosts fail cephadm check
            failed to probe daemons or devices

I even restarted server-1. No luck.

I'm on server-1. cephadm complains it cannot access to server-1. In basic term, server-1 cannot access server-1 (192.168.0.1)

server-1: 192.168.0.1
server-2: 192.168.0.3

$ ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1
> Success.

cephadm says server-1 is not reachable.  Why cephadm/ceph complain about this and what happend to my dashboard? I also redirected logs to file. I'm observing for any weird log input but unfortunately nothing helpful. Where should I look for? What am i missing?


So, when I ran this command:

$ ceph health detail

Output was:

HEALTH_WARN 1 hosts fail cephadm check; failed to probe daemons or devices
[WRN] CEPHADM_HOST_CHECK_FAILED: 1 hosts fail cephadm check
    host server-1 failed check: Failed to connect to server-1 (server-1).  Check that the host is reachable and accepts
connections using the cephadm SSH key
you may want to run:
> ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1
[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices
    host server-1 scrape failed: Failed to connect to server-1 (server-1).  Check that the host is reachable and accepts connections using the cephadm SSH key
you may want to run:
> ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1
    host server-1 ceph-volume inventory failed: Failed to connect
to server-1 (server-1).  Check that the host is reachable and accepts connections using the cephadm SSH key
you may want to run:
> ssh -F =(ceph cephadm get-ssh-config) -i =(ceph config-key get mgr/cephadm/ssh_identity_key) root@server-1

Thanks,
Gencer.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx