Thank you for taking time to reply to my issue. I have increased the log level to 10/10 for both the messenger and monitor debug and see the following pattern return in the logs. However I do not understand the severe high level log that is produced to deduct the problem. My I again ask for advice? Log output: 2019-10-18 10:58:28.962 7fd81fc02700 4 mon.mon4@-1(probing) e0 probe_timeout 0x55de1e9c51a0 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 bootstrap 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 sync_reset_requester 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 unregister_cluster_logger - not registered 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 cancel_probe_timeout (none scheduled) 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 _reset 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 cancel_probe_timeout (none scheduled) 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 timecheck_finish 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 scrub_event_cancel 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 scrub_reset 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 cancel_probe_timeout (none scheduled) 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 reset_probe_timeout 0x55de1e9c5260 after 2 seconds 2019-10-18 10:58:28.962 7fd81fc02700 10 mon.mon4@-1(probing) e0 probing other monitors 2019-10-18 10:58:28.962 7fd81fc02700 1 -- 10.200.1.104:6789/0 _send_message--> mon.0 10.200.1.101:6789/0 -- mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 -- ?+0 0x55de1e9e5400 2019-10-18 10:58:28.962 7fd81fc02700 1 -- 10.200.1.104:6789/0 --> 10.200.1.101:6789/0 -- mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 -- 0x55de1e9e5400 con 0 2019-10-18 10:58:28.962 7fd81fc02700 1 -- 10.200.1.104:6789/0 _send_message--> mon.1 10.200.1.102:6789/0 -- mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 -- ?+0 0x55de1e9e5680 2019-10-18 10:58:28.962 7fd81fc02700 1 -- 10.200.1.104:6789/0 --> 10.200.1.102:6789/0 -- mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 -- 0x55de1e9e5680 con 0 2019-10-18 10:58:28.962 7fd81fc02700 1 -- 10.200.1.104:6789/0 _send_message--> mon.2 10.200.1.103:6789/0 -- mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 -- ?+0 0x55de1e9e5900 2019-10-18 10:58:28.962 7fd81fc02700 1 -- 10.200.1.104:6789/0 --> 10.200.1.103:6789/0 -- mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 -- 0x55de1e9e5900 con 0 2019-10-18 10:58:28.962 7fd81abf8700 10 -- 10.200.1.104:6789/0 >> 10.200.1.101:6789/0 conn(0x55de1e7d3e00 :-1 s=STATE_OPEN pgs=2274435 cs=1 l=0).handle_write 2019-10-18 10:58:28.962 7fd819bf6700 10 -- 10.200.1.104:6789/0 >> 10.200.1.102:6789/0 conn(0x55de1e7d4400 :-1 s=STATE_OPEN pgs=2284339 cs=1 l=0).handle_write 2019-10-18 10:58:28.962 7fd81a3f7700 10 -- 10.200.1.104:6789/0 >> 10.200.1.103:6789/0 conn(0x55de1e7d4a00 :-1 s=STATE_OPEN pgs=2288108 cs=1 l=0).handle_write 2019-10-18 10:58:28.963 7fd81abf8700 10 -- 10.200.1.104:6789/0 >> 10.200.1.101:6789/0 conn(0x55de1e7d3e00 :-1 s=STATE_OPEN pgs=2274435 cs=1 l=0)._try_send sent bytes 136 remaining bytes 0 2019-10-18 10:58:28.963 7fd81abf8700 10 -- 10.200.1.104:6789/0 >> 10.200.1.101:6789/0 conn(0x55de1e7d3e00 :-1 s=STATE_OPEN pgs=2274435 cs=1 l=0).write_message sending 0x55de1e9e5400 done. 2019-10-18 10:58:28.963 7fd81a3f7700 10 -- 10.200.1.104:6789/0 >> 10.200.1.103:6789/0 conn(0x55de1e7d4a00 :-1 s=STATE_OPEN pgs=2288108 cs=1 l=0)._try_send sent bytes 136 remaining bytes 0 2019-10-18 10:58:28.963 7fd81a3f7700 10 -- 10.200.1.104:6789/0 >> 10.200.1.103:6789/0 conn(0x55de1e7d4a00 :-1 s=STATE_OPEN pgs=2288108 cs=1 l=0).write_message sending 0x55de1e9e5900 done. 2019-10-18 10:58:28.963 7fd819bf6700 10 -- 10.200.1.104:6789/0 >> 10.200.1.102:6789/0 conn(0x55de1e7d4400 :-1 s=STATE_OPEN pgs=2284339 cs=1 l=0)._try_send sent bytes 136 remaining bytes 0 2019-10-18 10:58:28.963 7fd819bf6700 10 -- 10.200.1.104:6789/0 >> 10.200.1.102:6789/0 conn(0x55de1e7d4400 :-1 s=STATE_OPEN pgs=2284339 cs=1 l=0).write_message sending 0x55de1e9e5680 done. 2019-10-18 10:58:28.963 7fd81abf8700 10 -- 10.200.1.104:6789/0 >> 10.200.1.101:6789/0 conn(0x55de1e7d3e00 :-1 s=STATE_OPEN_TAG_ACK pgs=2274435 cs=1 l=0).handle_ack got ack seq 20 >= 20 on 0x55de1e9e5400 mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 2019-10-18 10:58:28.963 7fd81a3f7700 10 -- 10.200.1.104:6789/0 >> 10.200.1.103:6789/0 conn(0x55de1e7d4a00 :-1 s=STATE_OPEN_TAG_ACK pgs=2288108 cs=1 l=0).handle_ack got ack seq 20 >= 20 on 0x55de1e9e5900 mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 2019-10-18 10:58:28.963 7fd819bf6700 10 -- 10.200.1.104:6789/0 >> 10.200.1.102:6789/0 conn(0x55de1e7d4400 :-1 s=STATE_OPEN_TAG_ACK pgs=2284339 cs=1 l=0).handle_ack got ack seq 20 >= 20 on 0x55de1e9e5680 mon_probe(probe aaf1547b-8944-4f48-b354-93659202c6fe name mon4 new) v6 2019-10-18 10:58:30.957 7fd81fc02700 -1 mon.mon4@-1(probing) e0 get_health_metrics reporting 4 slow ops, oldest is log(1 entries from seq 1 at 2019-10-18 10:57:53.085794) _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |