Hi All, I have ceph cluster with 3 monitors, 3 osd nodes (3 osds in each node) While Io was going on, rebooted a osd node which includes osds osd.6, osd.7, osd.8. osd.0 and osd.2 crashed with assert(e.version > info.last_update): PG:add_log_entry 2014-07-17 17:54:14.893962 7f91f3660700 -1 osd/PG.cc: In function 'void PG::add_log_entry(pg_log_entry_t&, ceph::bufferlist&)' thread 7f91f3660700 time 2014-07-17 17:54:13.252064 osd/PG.cc: 2619: FAILED assert(e.version > info.last_update) ceph version andisk-sprint-2-drop-3-390-g2dbd85c (2dbd85c94cf27a1ff0419c5ea9359af7fe30e9b6) 1: (PG::add_log_entry(pg_log_entry_t&, ceph::buffer::list&)+0x481) [0x733a61] 2: (PG::append_log(std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&, eversion_t, ObjectStore::Transaction&, bool)+0xdf) [0x74483f] 3: (ReplicatedBackend::sub_op_modify(std::tr1::shared_ptr<OpRequest>)+0xcfe) [0x8193be] 4: (ReplicatedBackend::handle_message(std::tr1::shared_ptr<OpRequest>)+0x4a6) [0x904586] 5: (ReplicatedPG::do_request(std::tr1::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x2db) [0x7aedcb] 6: (OSD::dequeue_op(boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x459) [0x635719] 7: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x346) [0x635ce6] 8: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x8ce) [0xa4a1ce] 9: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xa4c420] 10: (()+0x8182) [0x7f920f579182] 11: (clone()+0x6d) [0x7f920d91a30d] Raised tracker : http://tracker.ceph.com/issues/8887 Logs are attached to tracker. Thanks Sahana Lokeshappa Test Development Engineer I [cid:image001.png at 01CE9342.6D040E30] 3rd Floor, Bagmane Laurel, Bagmane Tech Park C V Raman nagar, Bangalore 560093 T: +918042422283 Sahana.Lokeshappa at SanDisk.com ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140721/e207dc66/attachment.htm> -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.png Type: image/png Size: 3296 bytes Desc: image003.png URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140721/e207dc66/attachment.png>
 |