I was able to catch on the crashes pn 951901 opn 0) v3 ==== 982012+0+0 (3546980463 0 0) 0x5e2ea00 con 0x3701420 -3> 2013-06-10 15:45:47.944085 7f27a5038700 20 mon.3@2(peon) e1 have connection -2> 2013-06-10 15:45:47.944092 7f27a5038700 20 mon.3@2(peon) e1 ms_dispatch existing session MonSession: mon.1 10.198.141.202:6789/0 is openallow * for mon.1 10.198.141.202:6789/0 -1> 2013-06-10 15:45:47.944100 7f27a5038700 20 mon.3@2(peon) e1 caps allow * 0> 2013-06-10 15:45:47.993974 7f27a5038700 -1 mon/Paxos.cc: In function 'void Paxos::handle_begin(MMonPaxos*)' thread 7f27a5038700 time 2013-06-10 15:45:47.944106 mon/Paxos.cc: 554: FAILED assert(begin->last_committed == last_committed) ceph version 0.61.3 (92b1e398576d55df8e5888dd1a9545ed3fd99532) 1: (Paxos::handle_begin(MMonPaxos*)+0x9cf) [0x4ddc3f] 2: (Paxos::dispatch(PaxosServiceMessage*)+0x25b) [0x4e269b] 3: (Monitor::_ms_dispatch(Message*)+0x10dc) [0x4b79ec] 4: (Monitor::ms_dispatch(Message*)+0x32) [0x4d1902] 5: (DispatchQueue::entry()+0x3f1) [0x6abae1] 6: (DispatchQueue::DispatchThread::entry()+0xd) [0x639bad] 7: (()+0x7e9a) [0x7f27aa62de9a] 8: (clone()+0x6d) [0x7f27a92c8ccd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 1/ 1 ms 20/20 mon 0/10 monc 0/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 hadoop 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -1/-1 (syslog threshold) 99/99 (stderr threshold) max_recent 10000 max_new 1000 log_file --- end dump of recent events --- 2013-06-10 15:45:50.148146 7f279d9df700 1 -- 10.198.141.203:6789/0 >> :/0 pipe(0x3830a00 sd=869 :6789 s=0 pgs=0 cs=0 l=0).accept sd=869 10.198.141.34:34566/0 2013-06-10 15:45:50.148355 7f279d9df700 10 mon.3@2(peon) e1 ms_verify_authorizer 10.198.141.34:6803/10847 osd protocol 0 terminate called after throwing an instance of 'ceph::FailedAssertion' *** Caught signal (Aborted) ** in thread 7f27a5038700 ceph version 0.61.3 (92b1e398576d55df8e5888dd1a9545ed3fd99532) 1: ceph-mon() [0x59255a] 2: (()+0xfcb0) [0x7f27aa635cb0] 3: (gsignal()+0x35) [0x7f27a920b425] 4: (abort()+0x17b) [0x7f27a920eb8b] 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f27a9b5d69d] 6: (()+0xb5846) [0x7f27a9b5b846] 7: (()+0xb5873) [0x7f27a9b5b873] 8: (()+0xb596e) [0x7f27a9b5b96e] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x649d6f] 10: (Paxos::handle_begin(MMonPaxos*)+0x9cf) [0x4ddc3f] 11: (Paxos::dispatch(PaxosServiceMessage*)+0x25b) [0x4e269b] 12: (Monitor::_ms_dispatch(Message*)+0x10dc) [0x4b79ec] 13: (Monitor::ms_dispatch(Message*)+0x32) [0x4d1902] 14: (DispatchQueue::entry()+0x3f1) [0x6abae1] 15: (DispatchQueue::DispatchThread::entry()+0xd) [0x639bad] 16: (()+0x7e9a) [0x7f27aa62de9a] 17: (clone()+0x6d) [0x7f27a92c8ccd] 2013-06-10 15:45:51.878383 7f27a5038700 -1 *** Caught signal (Aborted) ** in thread 7f27a5038700 ceph version 0.61.3 (92b1e398576d55df8e5888dd1a9545ed3fd99532) 1: ceph-mon() [0x59255a] 2: (()+0xfcb0) [0x7f27aa635cb0] 3: (gsignal()+0x35) [0x7f27a920b425] 4: (abort()+0x17b) [0x7f27a920eb8b] 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f27a9b5d69d] 6: (()+0xb5846) [0x7f27a9b5b846] 7: (()+0xb5873) [0x7f27a9b5b873] 8: (()+0xb596e) [0x7f27a9b5b96e] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x649d6f] 10: (Paxos::handle_begin(MMonPaxos*)+0x9cf) [0x4ddc3f] 11: (Paxos::dispatch(PaxosServiceMessage*)+0x25b) [0x4e269b] 12: (Monitor::_ms_dispatch(Message*)+0x10dc) [0x4b79ec] 13: (Monitor::ms_dispatch(Message*)+0x32) [0x4d1902] 14: (DispatchQueue::entry()+0x3f1) [0x6abae1] 15: (DispatchQueue::DispatchThread::entry()+0xd) [0x639bad] 16: (()+0x7e9a) [0x7f27aa62de9a] 17: (clone()+0x6d) [0x7f27a92c8ccd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- begin dump of recent events --- -2> 2013-06-10 15:45:50.148146 7f279d9df700 1 -- 10.198.141.203:6789/0 >> :/0 pipe(0x3830a00 sd=869 :6789 s=0 pgs=0 cs=0 l=0).accept sd=869 10.198.141.34:34566/0 -1> 2013-06-10 15:45:50.148355 7f279d9df700 10 mon.3@2(peon) e1 ms_verify_authorizer 10.198.141.34:6803/10847 osd protocol 0 0> 2013-06-10 15:45:51.878383 7f27a5038700 -1 *** Caught signal (Aborted) ** in thread 7f27a5038700 ceph version 0.61.3 (92b1e398576d55df8e5888dd1a9545ed3fd99532) 1: ceph-mon() [0x59255a] 2: (()+0xfcb0) [0x7f27aa635cb0] 3: (gsignal()+0x35) [0x7f27a920b425] 4: (abort()+0x17b) [0x7f27a920eb8b] 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f27a9b5d69d] 6: (()+0xb5846) [0x7f27a9b5b846] 7: (()+0xb5873) [0x7f27a9b5b873] 8: (()+0xb596e) [0x7f27a9b5b96e] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x649d6f] 10: (Paxos::handle_begin(MMonPaxos*)+0x9cf) [0x4ddc3f] 11: (Paxos::dispatch(PaxosServiceMessage*)+0x25b) [0x4e269b] 12: (Monitor::_ms_dispatch(Message*)+0x10dc) [0x4b79ec] 13: (Monitor::ms_dispatch(Message*)+0x32) [0x4d1902] 14: (DispatchQueue::entry()+0x3f1) [0x6abae1] 15: (DispatchQueue::DispatchThread::entry()+0xd) [0x639bad] 16: (()+0x7e9a) [0x7f27aa62de9a] 17: (clone()+0x6d) [0x7f27a92c8ccd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 1/ 1 ms 20/20 mon 0/10 monc 0/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 hadoop 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -1/-1 (syslog threshold) 99/99 (stderr threshold) max_recent 10000 max_new 1000 log_file --- end dump of recent events --- Nelson Jeppesen Disney Technology Solutions and Services Phone 206-588-5001 |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
