Hello Wyllys,
On Sat, Mar 17, 2018 at 6:37 AM, Wyllys Ingersoll
<wyllys.ingersoll@xxxxxxxxxxxxxx> wrote:
> Ubuntu 16.04.3
>
>
> One of my MDS servers keeps crashing and will not restart. The
> cluster has 3 MDS, the other 2 are up, but the first one will not
> restart. The logs are below. Any ideas what is wrong or how to get it
> back up and running?
You only use one active, correct?
> $ ceph -s
> cluster ba0c94fc-1168-11e6-aaea-000c290cc2d4
> health HEALTH_WARN
> mds cluster is degraded
> monmap e1: 3 mons at
> {mon01=10.16.51.21:6789/0,mon02=10.16.51.22:6789/0,mon03=10.16.51.23:6789/0}
> election epoch 352, quorum 0,1,2 mon01,mon02,mon03
> fsmap e18460: 1/1/1 up {0=mon03=up:replay}
> osdmap e427025: 93 osds: 93 up, 89 in
> flags sortbitwise,require_jewel_osds
> pgmap v51310487: 18960 pgs, 21 pools, 26329 GB data, 12939 kobjects
> 80586 GB used, 188 TB / 267 TB avail
> 18960 active+clean
> client io 0 B/s rd, 290 kB/s wr, 40 op/s rd, 87 op/s wr
>
>
>
> 2018-03-17 09:25:49.846771 7f425e3bf700 -1 *** Caught signal (Aborted) **
> in thread 7f425e3bf700 thread_name:md_log_replay
>
> ceph version 10.2.10 (5dc1e4c05cb68dbf62ae6fce3f0700e4654fdbbe)
> 1: (()+0x535e6e) [0x557481697e6e]
> 2: (()+0x11390) [0x7f426bfb6390]
> 3: (gsignal()+0x38) [0x7f426a39a428]
> 4: (abort()+0x16a) [0x7f426a39c02a]
> 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char
> const*)+0x26b) [0x5574817a0aab]
> 6: (EOpen::replay(MDSRank*)+0x75e) [0x55748167da0e]
> 7: (MDLog::_replay_thread()+0xe38) [0x5574815fa718]
> 8: (MDLog::ReplayThread::entry()+0xd) [0x5574813ac09d]
> 9: (()+0x76ba) [0x7f426bfac6ba]
> 10: (clone()+0x6d) [0x7f426a46c3dd]
> NOTE: a copy of the executable, or `objdump -rdS <executable>` is
> needed to interpret this.
This looks like: https://tracker.ceph.com/issues/21337
Are you using snapshots? The issue above was not backported to Jewel.
--
Patrick Donnelly
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
