So I first upgraded the mon, then went ahead and upgraded one of the osds which crashed and keeps crashing - probably when trying to upgrade the filestore. How should I proceed? FS is btrfs, one mon two osds. This is the conf for the osds: [osd] osd data = /srv/osd.$id osd journal = /srv/osd.$id.journal osd journal size = 1000 Here's the log output: root@ceph-osd-0:~# tail -F /var/log/ceph/ceph-osd.0.log --- end dump of recent events --- 2012-07-03 07:46:49.543030 7f2d0f332780 0 filestore(/srv/osd.0) mount FIEMAP ioctl is supported and appears to work 2012-07-03 07:46:49.543095 7f2d0f332780 0 filestore(/srv/osd.0) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option 2012-07-03 07:46:50.049179 7f2d0f332780 0 filestore(/srv/osd.0) mount detected btrfs 2012-07-03 07:46:50.049435 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs CLONE_RANGE ioctl is supported 2012-07-03 07:46:50.898083 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs SNAP_CREATE is supported 2012-07-03 07:46:50.937621 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs SNAP_DESTROY is supported 2012-07-03 07:46:51.164967 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs START_SYNC is supported (transid 44619) 2012-07-03 07:46:51.426019 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs WAIT_SYNC is supported 2012-07-03 07:46:51.664258 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs SNAP_CREATE_V2 is supported 2012-07-03 07:46:52.535058 7f2d0f332780 0 filestore(/srv/osd.0) mount syncfs(2) syscall fully supported (by glibc and kernel) 2012-07-03 07:46:52.535529 7f2d0f332780 -1 filestore(/srv/osd.0) FileStore::mount : stale version stamp detected: 2. Proceeding, do_update is set, performing disk format upgrade. 2012-07-03 07:46:52.535683 7f2d0f332780 0 filestore(/srv/osd.0) mount found snaps <650707,650708> 2012-07-03 07:47:04.317867 7f2d0f332780 0 filestore(/srv/osd.0) mount: enabling PARALLEL journal mode: btrfs, SNAP_CREATE_V2 detected and 'filestore btrfs snap' mode is enabled 2012-07-03 07:47:04.354557 7f2d0f332780 1 journal _open /srv/osd.0.journal fd 23: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0 2012-07-03 07:48:47.415664 7f2d0f332780 1 journal _open /srv/osd.0.journal fd 23: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0 2012-07-03 07:48:47.416522 7f2d0f332780 -1 FileStore is old at version 2. Updating... 2012-07-03 07:48:47.416535 7f2d0f332780 -1 Removing tmp pgs 2012-07-03 07:49:51.029035 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:49:56.029265 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:50:01.029413 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:50:06.029531 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:50:11.029651 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:50:16.029764 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:50:18.959270 7f2d08be4700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:50:19.273378 7f2d0f332780 -1 Getting collections 2012-07-03 07:50:19.273399 7f2d0f332780 -1 834 to process. 2012-07-03 07:50:19.274588 7f2d0f332780 -1 0/833 processed 2012-07-03 07:50:19.274651 7f2d0f332780 -1 Updating collection meta current version is 2 2012-07-03 07:51:21.031150 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:51:26.031309 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:51:31.031430 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:51:36.031592 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:51:41.031732 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:51:46.031878 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:51:51.032004 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:51:56.032134 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:52:01.032255 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:52:06.032381 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:52:09.361844 7f2d08be4700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 2012-07-03 07:53:11.033743 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:16.033859 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:21.034007 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:26.034140 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:31.034267 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:36.034398 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:41.034593 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:46.034735 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:51.034870 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:53:56.035015 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:01.035153 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:06.035289 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:11.035440 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:16.035581 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:21.035700 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:26.035839 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:31.035992 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:36.036139 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:41.036258 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:46.036394 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:51.036530 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:54:56.036651 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:55:01.036767 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:55:06.036911 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:55:11.037032 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 2012-07-03 07:55:11.037077 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had suicide timed out after 180 2012-07-03 07:55:11.038755 7f2d0bbea700 -1 common/HeartbeatMap.cc: In function 'bool ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, const char*, time_t)' thread 7f2d0bbea700 time 2012-07-03 07:55:11.037140 common/HeartbeatMap.cc: 78: FAILED assert(0 == "hit suicide timeout") ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030) 1: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x26a) [0x8285aa] 2: (ceph::HeartbeatMap::is_healthy()+0x87) [0x828d87] 3: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x828fc3] 4: (CephContextServiceThread::entry()+0x54) [0x7aa734] 5: (()+0x7e9a) [0x7f2d0e7c4e9a] 6: (clone()+0x6d) [0x7f2d0d4644bd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- begin dump of recent events --- -66> 2012-07-03 07:46:41.019588 7f2d0f332780 0 ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030), process ceph-osd, pid 3070 -65> 2012-07-03 07:46:49.543030 7f2d0f332780 0 filestore(/srv/osd.0) mount FIEMAP ioctl is supported and appears to work -64> 2012-07-03 07:46:49.543095 7f2d0f332780 0 filestore(/srv/osd.0) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option -63> 2012-07-03 07:46:50.049179 7f2d0f332780 0 filestore(/srv/osd.0) mount detected btrfs -62> 2012-07-03 07:46:50.049435 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs CLONE_RANGE ioctl is supported -61> 2012-07-03 07:46:50.898083 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs SNAP_CREATE is supported -60> 2012-07-03 07:46:50.937621 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs SNAP_DESTROY is supported -59> 2012-07-03 07:46:51.164967 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs START_SYNC is supported (transid 44619) -58> 2012-07-03 07:46:51.426019 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs WAIT_SYNC is supported -57> 2012-07-03 07:46:51.664258 7f2d0f332780 0 filestore(/srv/osd.0) mount btrfs SNAP_CREATE_V2 is supported -56> 2012-07-03 07:46:52.535058 7f2d0f332780 0 filestore(/srv/osd.0) mount syncfs(2) syscall fully supported (by glibc and kernel) -55> 2012-07-03 07:46:52.535529 7f2d0f332780 -1 filestore(/srv/osd.0) FileStore::mount : stale version stamp detected: 2. Proceeding, do_update is set, performing disk format upgrade. -54> 2012-07-03 07:46:52.535683 7f2d0f332780 0 filestore(/srv/osd.0) mount found snaps <650707,650708> -53> 2012-07-03 07:47:04.317867 7f2d0f332780 0 filestore(/srv/osd.0) mount: enabling PARALLEL journal mode: btrfs, SNAP_CREATE_V2 detected and 'filestore btrfs snap' mode is enabled -52> 2012-07-03 07:47:04.354557 7f2d0f332780 1 journal _open /srv/osd.0.journal fd 23: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0 -51> 2012-07-03 07:48:47.415664 7f2d0f332780 1 journal _open /srv/osd.0.journal fd 23: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0 -50> 2012-07-03 07:48:47.416522 7f2d0f332780 -1 FileStore is old at version 2. Updating... -49> 2012-07-03 07:48:47.416535 7f2d0f332780 -1 Removing tmp pgs -48> 2012-07-03 07:49:51.029035 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -47> 2012-07-03 07:49:56.029265 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -46> 2012-07-03 07:50:01.029413 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -45> 2012-07-03 07:50:06.029531 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -44> 2012-07-03 07:50:11.029651 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -43> 2012-07-03 07:50:16.029764 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -42> 2012-07-03 07:50:18.959270 7f2d08be4700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -41> 2012-07-03 07:50:19.273378 7f2d0f332780 -1 Getting collections -40> 2012-07-03 07:50:19.273399 7f2d0f332780 -1 834 to process. -39> 2012-07-03 07:50:19.274588 7f2d0f332780 -1 0/833 processed -38> 2012-07-03 07:50:19.274651 7f2d0f332780 -1 Updating collection meta current version is 2 -37> 2012-07-03 07:51:21.031150 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -36> 2012-07-03 07:51:26.031309 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -35> 2012-07-03 07:51:31.031430 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -34> 2012-07-03 07:51:36.031592 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -33> 2012-07-03 07:51:41.031732 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -32> 2012-07-03 07:51:46.031878 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -31> 2012-07-03 07:51:51.032004 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -30> 2012-07-03 07:51:56.032134 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -29> 2012-07-03 07:52:01.032255 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -28> 2012-07-03 07:52:06.032381 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -27> 2012-07-03 07:52:09.361844 7f2d08be4700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f2d08be4700' had timed out after 60 -26> 2012-07-03 07:53:11.033743 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -25> 2012-07-03 07:53:16.033859 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -24> 2012-07-03 07:53:21.034007 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -23> 2012-07-03 07:53:26.034140 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -22> 2012-07-03 07:53:31.034267 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -21> 2012-07-03 07:53:36.034398 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -20> 2012-07-03 07:53:41.034593 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -19> 2012-07-03 07:53:46.034735 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -18> 2012-07-03 07:53:51.034870 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -17> 2012-07-03 07:53:56.035015 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -16> 2012-07-03 07:54:01.035153 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -15> 2012-07-03 07:54:06.035289 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -14> 2012-07-03 07:54:11.035440 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -13> 2012-07-03 07:54:16.035581 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -12> 2012-07-03 07:54:21.035700 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -11> 2012-07-03 07:54:26.035839 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -10> 2012-07-03 07:54:31.035992 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -9> 2012-07-03 07:54:36.036139 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -8> 2012-07-03 07:54:41.036258 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -7> 2012-07-03 07:54:46.036394 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -6> 2012-07-03 07:54:51.036530 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -5> 2012-07-03 07:54:56.036651 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -4> 2012-07-03 07:55:01.036767 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -3> 2012-07-03 07:55:06.036911 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -2> 2012-07-03 07:55:11.037032 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had timed out after 60 -1> 2012-07-03 07:55:11.037077 7f2d0bbea700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f2d083e3700' had suicide timed out after 180 0> 2012-07-03 07:55:11.038755 7f2d0bbea700 -1 common/HeartbeatMap.cc: In function 'bool ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, const char*, time_t)' thread 7f2d0bbea700 time 2012-07-03 07:55:11.037140 common/HeartbeatMap.cc: 78: FAILED assert(0 == "hit suicide timeout") ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030) 1: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x26a) [0x8285aa] 2: (ceph::HeartbeatMap::is_healthy()+0x87) [0x828d87] 3: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x828fc3] 4: (CephContextServiceThread::entry()+0x54) [0x7aa734] 5: (()+0x7e9a) [0x7f2d0e7c4e9a] 6: (clone()+0x6d) [0x7f2d0d4644bd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- end dump of recent events --- 2012-07-03 07:55:11.043564 7f2d0bbea700 -1 *** Caught signal (Aborted) ** in thread 7f2d0bbea700 ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030) 1: /usr/bin/ceph-osd() [0x6e900a] 2: (()+0xfcb0) [0x7f2d0e7cccb0] 3: (gsignal()+0x35) [0x7f2d0d3a8445] 4: (abort()+0x17b) [0x7f2d0d3abbab] 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f2d0dcf669d] 6: (()+0xb5846) [0x7f2d0dcf4846] 7: (()+0xb5873) [0x7f2d0dcf4873] 8: (()+0xb596e) [0x7f2d0dcf496e] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x282) [0x79f662] 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x26a) [0x8285aa] 11: (ceph::HeartbeatMap::is_healthy()+0x87) [0x828d87] 12: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x828fc3] 13: (CephContextServiceThread::entry()+0x54) [0x7aa734] 14: (()+0x7e9a) [0x7f2d0e7c4e9a] 15: (clone()+0x6d) [0x7f2d0d4644bd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- begin dump of recent events --- 0> 2012-07-03 07:55:11.043564 7f2d0bbea700 -1 *** Caught signal (Aborted) ** in thread 7f2d0bbea700 ceph version 0.48argonaut (commit:c2b20ca74249892c8e5e40c12aa14446a2bf2030) 1: /usr/bin/ceph-osd() [0x6e900a] 2: (()+0xfcb0) [0x7f2d0e7cccb0] 3: (gsignal()+0x35) [0x7f2d0d3a8445] 4: (abort()+0x17b) [0x7f2d0d3abbab] 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f2d0dcf669d] 6: (()+0xb5846) [0x7f2d0dcf4846] 7: (()+0xb5873) [0x7f2d0dcf4873] 8: (()+0xb596e) [0x7f2d0dcf496e] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x282) [0x79f662] 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x26a) [0x8285aa] 11: (ceph::HeartbeatMap::is_healthy()+0x87) [0x828d87] 12: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x828fc3] 13: (CephContextServiceThread::entry()+0x54) [0x7aa734] 14: (()+0x7e9a) [0x7f2d0e7c4e9a] 15: (clone()+0x6d) [0x7f2d0d4644bd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- end dump of recent events --- -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html