HI , Dear cephers:
my lab env:
- ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
Yestoday , I restart all my OSD using systemctl restart ceph-osd.target and it stuck in fsck mount ,But I don't think about this.
Today, I set bluestore fsck on mount = false then restart again. Some osd be OK. but others goes wrong. I check its Log. As show me like below:
......
2018-02-01 14:16:37.622156 7ff8dd1cfd00 -1 log_channel(cluster) log [ERR] : 48.f1 log bound mismatch, info (tail,head] (9117'40641,9117'42184] actual [9112'1,9117'42184]
2018-02-01 14:16:37.668008 7ff8dd1cfd00 -1 log_channel(cluster) log [ERR] : 48.2f log bound mismatch, info (tail,head] (9117'47939,9117'49503] actual [9112'101,9117'49503]
2018-02-01 14:16:37.709207 7ff8dd1cfd00 -1 log_channel(cluster) log [ERR] : 48.1f8 log bound mismatch, info (tail,head] (9117'38544,9117'40116] actual [9112'1,9117'40116]
2018-02-01 14:16:37.753482 7ff8dd1cfd00 -1 log_channel(cluster) log [ERR] : 48.5f log bound mismatch, info (tail,head] (9117'47954,9117'49462] actual [9112'1,9117'49462]
2018-02-01 14:16:37.808295 7ff8dd1cfd00 -1 log_channel(cluster) log [ERR] : 48.b8 log bound mismatch, info (tail,head] (9117'70923,9117'72514] actual [9112'25601,9117'72514]
2018-02-01 14:16:37.854240 7ff8dd1cfd00 -1 log_channel(cluster) log [ERR] : 48.69 log bound mismatch, info (tail,head] (9117'44869,9117'46370] actual [9112'1,9117'46370]
2018-02-01 14:16:37.898908 7ff8dd1cfd00 -1 log_channel(cluster) log [ERR] : 48.1f log bound mismatch, info (tail,head] (9117'44119,9117'45683] actual [9112'1,9117'45683
--- end dump of recent events ---
2018-02-01 14:16:30.798901 7fc2a5b93700 -1 *** Caught signal (Aborted) **
in thread 7fc2a5b93700 thread_name:safe_timer
ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa29511) [0x7fc2d227d511]
2: (()+0xf370) [0x7fc2cee94370]
3: (gsignal()+0x37) [0x7fc2cdebe1d7]
4: (abort()+0x148) [0x7fc2cdebf8c8]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x284) [0x7fc2d22bc094]
6: (LogClient::_get_mon_log_message()+0xb04) [0x7fc2d228d344]
7: (LogClient::get_mon_log_message(bool)+0x44) [0x7fc2d228d3c4]
8: (MonClient::send_log(bool)+0x19) [0x7fc2d22cf0e9]
9: (MonClient::tick()+0x402) [0x7fc2d22d9db2]
10: (Context::complete(int)+0x9) [0x7fc2d1d6f0c9]
11: (SafeTimer::timer_thread()+0x104) [0x7fc2d22b71a4]
12: (SafeTimerThread::entry()+0xd) [0x7fc2d22b8bcd]
13: (()+0x7dc5) [0x7fc2cee8cdc5]
14: (clone()+0x6d) [0x7fc2cdf8073d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
--- begin dump of recent events ---
0> 2018-02-01 14:16:30.798901 7fc2a5b93700 -1 *** Caught signal (Aborted) **
in thread 7fc2a5b93700 thread_name:safe_timer
ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (()+0xa29511) [0x7fc2d227d511]
2: (()+0xf370) [0x7fc2cee94370]
3: (gsignal()+0x37) [0x7fc2cdebe1d7]
4: (abort()+0x148) [0x7fc2cdebf8c8]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x284) [0x7fc2d22bc094]
6: (LogClient::_get_mon_log_message()+0xb04) [0x7fc2d228d344]
7: (LogClient::get_mon_log_message(bool)+0x44) [0x7fc2d228d3c4]
8: (MonClient::send_log(bool)+0x19) [0x7fc2d22cf0e9]
9: (MonClient::tick()+0x402) [0x7fc2d22d9db2]
10: (Context::complete(int)+0x9) [0x7fc2d1d6f0c9]
11: (SafeTimer::timer_thread()+0x104) [0x7fc2d22b71a4]
12: (SafeTimerThread::entry()+0xd) [0x7fc2d22b8bcd]
13: (()+0x7dc5) [0x7fc2cee8cdc5]
14: (clone()+0x6d) [0x7fc2cdf8073d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
....
What can I do to get the crash OSD being OK (up ) ? Or ,I can only destroy the bad osd and re-add to cluster ?
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
