Hi List,
FYI
I tripped across a bug when adding a new monitor into an existing
cluster
(see attached). I was on GIT commit
4b3bb5ab37a05fa001d59f24da7d9c30d650321b.
I found that the function 'bool Paxos::is_consistent()' was added in
commit
0bd545f5d6beb790e67fe2ee61f57d2ed6332531.
I reset back to commit 3a7bb9999b49641ee28cac91e493aa4c231ede54 (it's
parent) and it has gone away.
--
Thanks,
Dyweni
2012-02-17 11:39:07.797739 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) mount
2012-02-17 11:39:07.797785 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) reading at off 0 of 21
2012-02-17 11:39:07.797792 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl magic = 21 bytes
2012-02-17 11:39:07.797800 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) reading at off 0 of 75
2012-02-17 11:39:07.797803 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl feature_set = 75 bytes
2012-02-17 11:39:07.797818 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl monmap/latest No such file or directory
2012-02-17 11:39:07.797823 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) reading at off 0 of 588
2012-02-17 11:39:07.797826 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl mkfs/monmap = 588 bytes
2012-02-17 11:39:07.797851 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3 does not exist in monmap, will attempt to join an existing cluster
[0;33m ** WARNING: Ceph is still under development. Any feedback can be directed **[0m
[0;33m ** at ceph-devel@xxxxxxxxxxxxxxx or http://ceph.newdream.net/. **[0m
2012-02-17 11:39:07.798028 b6efc710 ceph version 0.41-369-g4b3bb5a (commit:4b3bb5ab37a05fa001d59f24da7d9c30d650321b), process ceph-mon, pid 1125
starting mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3 rank -1 at 192.168.0.4:6789/0 mon_data /ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata fsid 73d6c057-d1dd-4e30-85b3-e87a2f97fa3f
2012-02-17 11:39:07.798256 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing).pg v0 update_full_ratios full 0.95 nearfull 0.85
2012-02-17 11:39:07.798313 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing) e1 init fsid 73d6c057-d1dd-4e30-85b3-e87a2f97fa3f
2012-02-17 11:39:07.798336 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl pgmap/latest No such file or directory
2012-02-17 11:39:07.798350 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int mdsmap/last_committed = 6
2012-02-17 11:39:07.798355 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int mdsmap/first_committed = 1
2012-02-17 11:39:07.798361 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) reading at off 0 of 1036
2012-02-17 11:39:07.798365 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl mdsmap/latest = 1036 bytes
2012-02-17 11:39:07.798373 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int mdsmap/slurping = 0
2012-02-17 11:39:07.798379 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing).mds e0 update_from_paxos paxosv 6, my e 0
2012-02-17 11:39:07.798385 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) reading at off 0 of 1024
2012-02-17 11:39:07.798389 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl mdsmap/6 = 1024 bytes
2012-02-17 11:39:07.798392 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing).mds e0 update_from_paxos got 6
2012-02-17 11:39:07.798429 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing).mds e6 new map
2012-02-17 11:39:07.798463 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing).mds e6 print_map
epoch 6
flags 0
created 2012-02-17 11:19:48.979439
modified 2012-02-17 11:20:14.357436
tableserver 0
root 0
session_timeout 60
session_autoclose 300
last_failure 0
last_failure_osd_epoch 0
compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object}
max_mds 1
in 0
up {0=4098}
failed
stopped
data_pools [0]
metadata_pool 1
4098: 192.168.0.2:6800/1228 '24c94c97f0d6d131ee1ad7173d0f78b350d41ac1' mds.0.1 up:active seq 4
4099: 192.168.0.3:6800/1223 '3be17740a935bde47911960f1a6bd28f947b2391' mds.-1.0 up:standby seq 1
4105: 192.168.0.1:6800/1866 '85ffeac8368302141b36f2500e0e96289bd9e6cd' mds.-1.0 up:standby seq 1
2012-02-17 11:39:07.798469 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing).mds e6 update_logger
2012-02-17 11:39:07.798480 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl osdmap/latest No such file or directory
2012-02-17 11:39:07.798490 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int logm/last_committed = 149
2012-02-17 11:39:07.798495 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int logm/first_committed = 1
2012-02-17 11:39:07.798501 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) reading at off 0 of 9328
2012-02-17 11:39:07.798509 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl logm/latest = 9328 bytes
2012-02-17 11:39:07.798515 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int logm/slurping = 0
2012-02-17 11:39:07.798521 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) reading at off 0 of 9328
2012-02-17 11:39:07.798525 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl logm/latest = 9328 bytes
2012-02-17 11:39:07.798530 b6efc710 mon.2a597b6ca798c9cd55a43279598b81a328df7a3c_3@-1(probing).log v149 update_from_paxos loading summary e149
2012-02-17 11:39:07.798722 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int monmap/last_committed = 1
2012-02-17 11:39:07.798727 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int monmap/first_committed = 1
2012-02-17 11:39:07.798731 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_bl monmap/latest No such file or directory
2012-02-17 11:39:07.798736 b6efc710 store(/ceph/mon/disk7dc86aa3e88a9a2def2153a3f6c127e44deec449/mondata) get_int monmap/slurping = 0
mon/Paxos.cc: In function 'bool Paxos::is_consistent()' thread b6efc710 time 2012-02-17 11:39:07.798742
mon/Paxos.cc: 1081: FAILED assert(consistent || (slurping == 1))
ceph version 0.41-369-g4b3bb5a (commit:4b3bb5ab37a05fa001d59f24da7d9c30d650321b)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0xa4) [0x84377e5]
2: (Paxos::is_consistent()+0xd8) [0x82e066e]
3: (Monitor::init()+0x616) [0x82a4bd0]
4: (main()+0x1edf) [0x82969da]
5: (__libc_start_main()+0xe6) [0xb6f18296]
6: ceph-mon() [0x8294931]
ceph version 0.41-369-g4b3bb5a (commit:4b3bb5ab37a05fa001d59f24da7d9c30d650321b)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0xa4) [0x84377e5]
2: (Paxos::is_consistent()+0xd8) [0x82e066e]
3: (Monitor::init()+0x616) [0x82a4bd0]
4: (main()+0x1edf) [0x82969da]
5: (__libc_start_main()+0xe6) [0xb6f18296]
6: ceph-mon() [0x8294931]
terminate called after throwing an instance of 'ceph::FailedAssertion'
*** Caught signal (Aborted) **
in thread b6efc710
ceph version 0.41-369-g4b3bb5a (commit:4b3bb5ab37a05fa001d59f24da7d9c30d650321b)
1: (ceph::BackTrace::BackTrace(int)+0x30) [0x8437b44]
2: ceph-mon() [0x8484796]
3: [0xb7785400]
4: [0xb7785424]
5: (gsignal()+0x51) [0xb6f2c171]
6: (abort()+0x17e) [0xb6f2d94e]
7: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0xb7150d85]
8: (()+0xafc85) [0xb714ec85]
9: (()+0xafcc2) [0xb714ecc2]
10: (()+0xafe31) [0xb714ee31]
11: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x21e) [0x843795f]
12: (Paxos::is_consistent()+0xd8) [0x82e066e]
13: (Monitor::init()+0x616) [0x82a4bd0]
14: (main()+0x1edf) [0x82969da]
15: (__libc_start_main()+0xe6) [0xb6f18296]
16: ceph-mon() [0x8294931]
