Nope. Straight from 0.87 to 0.94.1. FWIW, at someone's suggestion, I just upgraded the kernel on one of the boxes from 3.14 to 3.18; no improvement. Rebooting didn't help, either. Still failing with the same error in the logs.
On Sun, Apr 19, 2015 at 2:06 PM Robert LeBlanc <robert@xxxxxxxxxxxxx> wrote:
Did you upgrade from 0.92? If you did, did you flush the logs before upgrading?On Sun, Apr 19, 2015 at 1:02 PM, Scott Laird <scott@xxxxxxxxxxx> wrote:I'm upgrading from Giant to Hammer (0.94.1), and I'm seeing a ton of OSDs die (and stay dead) with this error in the logs:2015-04-19 11:53:36.796847 7f61fa900900 -1 osd/OSD.h: In function 'OSDMapRef OSDService::get_map(epoch_t)' thread 7f61fa900900 time 2015-04-19 11:53:36.794951osd/OSD.h: 716: FAILED assert(ret)ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0xbc271b]2: (OSDService::get_map(unsigned int)+0x3f) [0x70923f]3: (OSD::load_pgs()+0x1769) [0x6c35d9]4: (OSD::init()+0x71f) [0x6c4c7f]5: (main()+0x2860) [0x651fc0]6: (__libc_start_main()+0xf5) [0x7f61f7a3fec5]7: /usr/bin/ceph-osd() [0x66aff7]NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.This is on a small cluster, with ~40 OSDs on 5 servers running Ubuntu 14.04. So far, every single server that I've upgraded has had at least one disk that has failed to restart with this error, and one has had several disks in this state.Restarting the OSD after it dies with this doesn't help.I haven't lost any data through this due to my slow rollout, but it's really annoying.Here are two full logs from OSDs on two different machines:Any suggestions?Scott_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
