lost all monitors at the same time

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

I accidently lost power on all nodes at the same time and now my monitors
do not get in quorum anymore. I cannot mount my cephfs anymore, I cannot
see the status anymore, seems like all data is lost.

I used cephadm to deploy all nodes, I only see the monitor docker container
on 2 hosts,
and both only log "e18 handle_auth_request failed to assign global_id".

Most ceph commands are just stuck, but using the admin socket I at least
get the status from those 2 monitors.

Do I need to rebuild my monmap from the OSDs now or is my data gone? I
don't understand the exact issue here. ceph orch commands are stuck and I
cannot see if any services are missing


sudo ceph --admin-daemon
/run/ceph/69f993e2-637e-11ed-8ca1-22aca0408523/ceph-mon.flucky-server.asok
mon_status

{
    "name": "flucky-server",
    "rank": 0,
    "state": "probing",
    "election_epoch": 0,
    "quorum": [],
    "features": {
        "required_con": "2449958755906961412",
        "required_mon": [
            "kraken",
            "luminous",
            "mimic",
            "osdmap-prune",
            "nautilus",
            "octopus",
            "pacific",
            "elector-pinging",
            "quincy"
        ],
        "quorum_con": "0",
        "quorum_mon": []
    },
    "outside_quorum": [
        "flucky-server",
        "matrixpi"
    ],
    "extra_probe_peers": [],
    "sync_provider": [],
    "monmap": {
        "epoch": 18,
        "fsid": "69f993e2-637e-11ed-8ca1-22aca0408523",
        "modified": "2022-11-15T22:36:40.883664Z",
        "created": "2022-11-13T18:10:17.638340Z",
        "min_mon_release": 17,
        "min_mon_release_name": "quincy",
        "election_strategy": 1,
        "disallowed_leaders: ": "",
        "stretch_mode": false,
        "tiebreaker_mon": "",
        "features": {
            "persistent": [
                "kraken",
                "luminous",
                "mimic",
                "osdmap-prune",
                "nautilus",
                "octopus",
                "pacific",
                "elector-pinging",
                "quincy"
            ],
            "optional": []
        },
        "mons": [
            {
                "rank": 0,
                "name": "flucky-server",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.2:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.2:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.2:6789/0",
                "public_addr": "192.168.0.2:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            },
            {
                "rank": 1,
                "name": "matrixpi",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.3:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.3:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.3:6789/0",
                "public_addr": "192.168.0.3:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            },
            {
                "rank": 2,
                "name": "raspberrypi3-09",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.29:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.29:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.29:6789/0",
                "public_addr": "192.168.0.29:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            },
            {
                "rank": 3,
                "name": "raspberrypi3-02",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.22:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.22:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.22:6789/0",
                "public_addr": "192.168.0.22:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            }
        ]
    },
    "feature_map": {
        "mon": [
            {
                "features": "0x3f01cfbf7ffdffff",
                "release": "luminous",
                "num": 1
            }
        ]
    },
    "stretch_mode": false
}



[feedc0de@matrixpi ~]$ sudo ceph --admin-daemon
/run/ceph/69f993e2-637e-11ed-8ca1-22aca0408523/ceph-mon.matrixpi.asok
mon_status

{
    "name": "matrixpi",
    "rank": 1,
    "state": "probing",
    "election_epoch": 0,
    "quorum": [],
    "features": {
        "required_con": "2449958755906961412",
        "required_mon": [
            "kraken",
            "luminous",
            "mimic",
            "osdmap-prune",
            "nautilus",
            "octopus",
            "pacific",
            "elector-pinging",
            "quincy"
        ],
        "quorum_con": "0",
        "quorum_mon": []
    },
    "outside_quorum": [
        "flucky-server",
        "matrixpi"
    ],
    "extra_probe_peers": [],
    "sync_provider": [],
    "monmap": {
        "epoch": 18,
        "fsid": "69f993e2-637e-11ed-8ca1-22aca0408523",
        "modified": "2022-11-15T22:36:40.883664Z",
        "created": "2022-11-13T18:10:17.638340Z",
        "min_mon_release": 17,
        "min_mon_release_name": "quincy",
        "election_strategy": 1,
        "disallowed_leaders: ": "",
        "stretch_mode": false,
        "tiebreaker_mon": "",
        "features": {
            "persistent": [
                "kraken",
                "luminous",
                "mimic",
                "osdmap-prune",
                "nautilus",
                "octopus",
                "pacific",
                "elector-pinging",
                "quincy"
            ],
            "optional": []
        },
        "mons": [
            {
                "rank": 0,
                "name": "flucky-server",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.2:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.2:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.2:6789/0",
                "public_addr": "192.168.0.2:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            },
            {
                "rank": 1,
                "name": "matrixpi",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.3:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.3:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.3:6789/0",
                "public_addr": "192.168.0.3:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            },
            {
                "rank": 2,
                "name": "raspberrypi3-09",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.29:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.29:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.29:6789/0",
                "public_addr": "192.168.0.29:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            },
            {
                "rank": 3,
                "name": "raspberrypi3-02",
                "public_addrs": {
                    "addrvec": [
                        {
                            "type": "v2",
                            "addr": "192.168.0.22:3300",
                            "nonce": 0
                        },
                        {
                            "type": "v1",
                            "addr": "192.168.0.22:6789",
                            "nonce": 0
                        }
                    ]
                },
                "addr": "192.168.0.22:6789/0",
                "public_addr": "192.168.0.22:6789/0",
                "priority": 0,
                "weight": 0,
                "crush_location": "{}"
            }
        ]
    },
    "feature_map": {
        "mon": [
            {
                "features": "0x3f01cfbf7ffdffff",
                "release": "luminous",
                "num": 1
            }
        ],
        "client": [
            {
                "features": "0x2f018fb87aa4aafe",
                "release": "luminous",
                "num": 1
            }
        ]
    },
    "stretch_mode": false
}


 feedc0de@flucky-server:~$ docker logs -f
ceph-69f993e2-637e-11ed-8ca1-22aca0408523-mon-flucky-server
debug 2022-11-15T23:36:06.582+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:07.079+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:07.179+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:07.289+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:07.429+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:07.899+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:08.632+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:09.785+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:10.282+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:10.632+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:11.152+0000 7f0b249fe700  0 log_channel(audit) log
[DBG] : from='admin socket' entity='admin socket' cmd='mon_status' args=[]:
dispatch
debug 2022-11-15T23:36:11.152+0000 7f0b249fe700  0 log_channel(audit) log
[DBG] : from='admin socket' entity='admin socket' cmd=mon_status args=[]:
finished
debug 2022-11-15T23:36:11.642+0000 7f0b1c9ee700  0 cephx: verify_authorizer
could not decrypt ticket info: error: bad magic in decode_decrypt,
16288580011235018902 != 18374858748799134293
debug 2022-11-15T23:36:13.632+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:16.192+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:16.689+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:17.039+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:18.632+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:22.233+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.349+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.959+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:23.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:26.659+0000 7f0b1c9ee700  0 cephx: verify_authorizer
could not decrypt ticket info: error: bad magic in decode_decrypt,
16288580011235018902 != 18374858748799134293
debug 2022-11-15T23:36:28.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:29.006+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:29.503+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:29.853+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:33.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:37.286+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:37.410+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:38.020+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:38.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:41.667+0000 7f0b1c9ee700  0 cephx: verify_authorizer
could not decrypt ticket info: error: bad magic in decode_decrypt,
16288580011235018902 != 18374858748799134293
debug 2022-11-15T23:36:43.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:44.020+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:44.517+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:44.867+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:48.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:52.340+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:52.460+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:53.077+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:53.634+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:56.687+0000 7f0b1c9ee700  0 cephx: verify_authorizer
could not decrypt ticket info: error: bad magic in decode_decrypt,
16288580011235018902 != 18374858748799134293
debug 2022-11-15T23:36:58.634+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:59.037+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:59.534+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:59.881+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:03.634+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
bytes epoch 0)



[feedc0de@matrixpi ~]$ docker logs -f
ceph-69f993e2-637e-11ed-8ca1-22aca0408523-mon-matrixpi
debug 2022-11-15T23:36:11.467+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:16.081+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:16.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:16.690+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:21.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:22.369+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.390+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.476+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.575+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.718+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.820+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:22.825+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:26.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:28.894+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:29.502+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:31.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:36.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:37.391+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:37.445+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:37.528+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:37.624+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:37.771+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:37.857+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:37.869+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:41.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:43.909+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:44.518+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:46.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:51.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:52.450+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:52.502+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:52.577+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:52.684+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:52.821+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:52.916+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:52.918+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:56.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:36:58.926+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:36:59.533+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:01.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:37:06.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:37:07.501+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:07.546+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:07.635+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:07.743+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:07.875+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:07.975+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:07.977+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:11.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:37:13.938+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:14.546+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
e18 handle_auth_request failed to assign global_id
debug 2022-11-15T23:37:16.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
debug 2022-11-15T23:37:21.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
bytes epoch 0)
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux