Re: lost all monitors at the same time

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



hi once again,

what does this error message "e18 handle_auth_request failed to assign
global_id" actually mean?

is this an indication that I need to manually rebuild the monmap from the
OSDs?

thanks,
daniel

Am Mi., 16. Nov. 2022 um 00:39 Uhr schrieb Daniel Brunner
<daniel@brunner.ninja>:

> Hi,
>
> I accidently lost power on all nodes at the same time and now my monitors
> do not get in quorum anymore. I cannot mount my cephfs anymore, I cannot
> see the status anymore, seems like all data is lost.
>
> I used cephadm to deploy all nodes, I only see the monitor docker
> container on 2 hosts,
> and both only log "e18 handle_auth_request failed to assign global_id".
>
> Most ceph commands are just stuck, but using the admin socket I at least
> get the status from those 2 monitors.
>
> Do I need to rebuild my monmap from the OSDs now or is my data gone? I
> don't understand the exact issue here. ceph orch commands are stuck and I
> cannot see if any services are missing
>
>
> sudo ceph --admin-daemon
> /run/ceph/69f993e2-637e-11ed-8ca1-22aca0408523/ceph-mon.flucky-server.asok
> mon_status
>
> {
>     "name": "flucky-server",
>     "rank": 0,
>     "state": "probing",
>     "election_epoch": 0,
>     "quorum": [],
>     "features": {
>         "required_con": "2449958755906961412",
>         "required_mon": [
>             "kraken",
>             "luminous",
>             "mimic",
>             "osdmap-prune",
>             "nautilus",
>             "octopus",
>             "pacific",
>             "elector-pinging",
>             "quincy"
>         ],
>         "quorum_con": "0",
>         "quorum_mon": []
>     },
>     "outside_quorum": [
>         "flucky-server",
>         "matrixpi"
>     ],
>     "extra_probe_peers": [],
>     "sync_provider": [],
>     "monmap": {
>         "epoch": 18,
>         "fsid": "69f993e2-637e-11ed-8ca1-22aca0408523",
>         "modified": "2022-11-15T22:36:40.883664Z",
>         "created": "2022-11-13T18:10:17.638340Z",
>         "min_mon_release": 17,
>         "min_mon_release_name": "quincy",
>         "election_strategy": 1,
>         "disallowed_leaders: ": "",
>         "stretch_mode": false,
>         "tiebreaker_mon": "",
>         "features": {
>             "persistent": [
>                 "kraken",
>                 "luminous",
>                 "mimic",
>                 "osdmap-prune",
>                 "nautilus",
>                 "octopus",
>                 "pacific",
>                 "elector-pinging",
>                 "quincy"
>             ],
>             "optional": []
>         },
>         "mons": [
>             {
>                 "rank": 0,
>                 "name": "flucky-server",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.2:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.2:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.2:6789/0",
>                 "public_addr": "192.168.0.2:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             },
>             {
>                 "rank": 1,
>                 "name": "matrixpi",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.3:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.3:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.3:6789/0",
>                 "public_addr": "192.168.0.3:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             },
>             {
>                 "rank": 2,
>                 "name": "raspberrypi3-09",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.29:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.29:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.29:6789/0",
>                 "public_addr": "192.168.0.29:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             },
>             {
>                 "rank": 3,
>                 "name": "raspberrypi3-02",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.22:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.22:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.22:6789/0",
>                 "public_addr": "192.168.0.22:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             }
>         ]
>     },
>     "feature_map": {
>         "mon": [
>             {
>                 "features": "0x3f01cfbf7ffdffff",
>                 "release": "luminous",
>                 "num": 1
>             }
>         ]
>     },
>     "stretch_mode": false
> }
>
>
>
> [feedc0de@matrixpi ~]$ sudo ceph --admin-daemon
> /run/ceph/69f993e2-637e-11ed-8ca1-22aca0408523/ceph-mon.matrixpi.asok
> mon_status
>
> {
>     "name": "matrixpi",
>     "rank": 1,
>     "state": "probing",
>     "election_epoch": 0,
>     "quorum": [],
>     "features": {
>         "required_con": "2449958755906961412",
>         "required_mon": [
>             "kraken",
>             "luminous",
>             "mimic",
>             "osdmap-prune",
>             "nautilus",
>             "octopus",
>             "pacific",
>             "elector-pinging",
>             "quincy"
>         ],
>         "quorum_con": "0",
>         "quorum_mon": []
>     },
>     "outside_quorum": [
>         "flucky-server",
>         "matrixpi"
>     ],
>     "extra_probe_peers": [],
>     "sync_provider": [],
>     "monmap": {
>         "epoch": 18,
>         "fsid": "69f993e2-637e-11ed-8ca1-22aca0408523",
>         "modified": "2022-11-15T22:36:40.883664Z",
>         "created": "2022-11-13T18:10:17.638340Z",
>         "min_mon_release": 17,
>         "min_mon_release_name": "quincy",
>         "election_strategy": 1,
>         "disallowed_leaders: ": "",
>         "stretch_mode": false,
>         "tiebreaker_mon": "",
>         "features": {
>             "persistent": [
>                 "kraken",
>                 "luminous",
>                 "mimic",
>                 "osdmap-prune",
>                 "nautilus",
>                 "octopus",
>                 "pacific",
>                 "elector-pinging",
>                 "quincy"
>             ],
>             "optional": []
>         },
>         "mons": [
>             {
>                 "rank": 0,
>                 "name": "flucky-server",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.2:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.2:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.2:6789/0",
>                 "public_addr": "192.168.0.2:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             },
>             {
>                 "rank": 1,
>                 "name": "matrixpi",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.3:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.3:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.3:6789/0",
>                 "public_addr": "192.168.0.3:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             },
>             {
>                 "rank": 2,
>                 "name": "raspberrypi3-09",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.29:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.29:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.29:6789/0",
>                 "public_addr": "192.168.0.29:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             },
>             {
>                 "rank": 3,
>                 "name": "raspberrypi3-02",
>                 "public_addrs": {
>                     "addrvec": [
>                         {
>                             "type": "v2",
>                             "addr": "192.168.0.22:3300",
>                             "nonce": 0
>                         },
>                         {
>                             "type": "v1",
>                             "addr": "192.168.0.22:6789",
>                             "nonce": 0
>                         }
>                     ]
>                 },
>                 "addr": "192.168.0.22:6789/0",
>                 "public_addr": "192.168.0.22:6789/0",
>                 "priority": 0,
>                 "weight": 0,
>                 "crush_location": "{}"
>             }
>         ]
>     },
>     "feature_map": {
>         "mon": [
>             {
>                 "features": "0x3f01cfbf7ffdffff",
>                 "release": "luminous",
>                 "num": 1
>             }
>         ],
>         "client": [
>             {
>                 "features": "0x2f018fb87aa4aafe",
>                 "release": "luminous",
>                 "num": 1
>             }
>         ]
>     },
>     "stretch_mode": false
> }
>
>
>  feedc0de@flucky-server:~$ docker logs -f
> ceph-69f993e2-637e-11ed-8ca1-22aca0408523-mon-flucky-server
> debug 2022-11-15T23:36:06.582+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:07.079+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:07.179+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:07.289+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:07.429+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:07.899+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:08.632+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:09.785+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:10.282+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:10.632+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:11.152+0000 7f0b249fe700  0 log_channel(audit) log
> [DBG] : from='admin socket' entity='admin socket' cmd='mon_status' args=[]:
> dispatch
> debug 2022-11-15T23:36:11.152+0000 7f0b249fe700  0 log_channel(audit) log
> [DBG] : from='admin socket' entity='admin socket' cmd=mon_status args=[]:
> finished
> debug 2022-11-15T23:36:11.642+0000 7f0b1c9ee700  0 cephx:
> verify_authorizer could not decrypt ticket info: error: bad magic in
> decode_decrypt, 16288580011235018902 != 18374858748799134293
> debug 2022-11-15T23:36:13.632+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:16.192+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:16.689+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:17.039+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:18.632+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:22.233+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.349+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.959+0000 7f0b1d1ef700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:23.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:26.659+0000 7f0b1c9ee700  0 cephx:
> verify_authorizer could not decrypt ticket info: error: bad magic in
> decode_decrypt, 16288580011235018902 != 18374858748799134293
> debug 2022-11-15T23:36:28.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:29.006+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:29.503+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:29.853+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:33.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:37.286+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:37.410+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:38.020+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:38.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 2 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:41.667+0000 7f0b1c9ee700  0 cephx:
> verify_authorizer could not decrypt ticket info: error: bad magic in
> decode_decrypt, 16288580011235018902 != 18374858748799134293
> debug 2022-11-15T23:36:43.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:44.020+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:44.517+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:44.867+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:48.633+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:52.340+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:52.460+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:53.077+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:53.634+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:56.687+0000 7f0b1c9ee700  0 cephx:
> verify_authorizer could not decrypt ticket info: error: bad magic in
> decode_decrypt, 16288580011235018902 != 18374858748799134293
> debug 2022-11-15T23:36:58.634+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:59.037+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:59.534+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:59.881+0000 7f0b1c9ee700  1 mon.flucky-server@0(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:03.634+0000 7f0b211f7700 -1 mon.flucky-server@0(probing)
> e18 get_health_metrics reporting 4 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
>
>
>
> [feedc0de@matrixpi ~]$ docker logs -f
> ceph-69f993e2-637e-11ed-8ca1-22aca0408523-mon-matrixpi
> debug 2022-11-15T23:36:11.467+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:16.081+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:16.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:16.690+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:21.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:22.369+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.390+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.476+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.575+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.718+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.820+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:22.825+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:26.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:28.894+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:29.502+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:31.468+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:36.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:37.391+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:37.445+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:37.528+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:37.624+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:37.771+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:37.857+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:37.869+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:41.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:43.909+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:44.518+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:46.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:51.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:52.450+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:52.502+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:52.577+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:52.684+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:52.821+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:52.916+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:52.918+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:56.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:36:58.926+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:36:59.533+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:01.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:37:06.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:37:07.501+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:07.546+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:07.635+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:07.743+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:07.875+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:07.975+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:07.977+0000 ffff9fe6d080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:11.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:37:13.938+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:14.546+0000 ffff9f66c080  1 mon.matrixpi@1(probing)
> e18 handle_auth_request failed to assign global_id
> debug 2022-11-15T23:37:16.469+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
> debug 2022-11-15T23:37:21.470+0000 ffffa3e75080 -1 mon.matrixpi@1(probing)
> e18 get_health_metrics reporting 7 slow ops, oldest is auth(proto 0 34
> bytes epoch 0)
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux