I'll see what I can do with this tomorrow, but it can be difficult to deal with commits from an out-of-tree build, or even with commits that got merged in following other changes (which is what happened with this commit). I didn't see any obviously relevant commits in the git history, so I want to track it down. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Mon, Jul 21, 2014 at 6:35 AM, Sahana Lokeshappa < Sahana.Lokeshappa at sandisk.com> wrote: > Hi All, > > > > I have ceph cluster with 3 monitors, 3 osd nodes (3 osds in each node) > > > > While Io was going on, rebooted a osd node which includes osds osd.6, > osd.7, osd.8. > > > > osd.0 and osd.2 crashed with assert(e.version > info.last_update): > PG:add_log_entry > > > > 2014-07-17 17:54:14.893962 7f91f3660700 -1 osd/PG.cc: In function 'void > PG::add_log_entry(pg_log_entry_t&, ceph::bufferlist&)' thread 7f91f3660700 > time 2014-07-17 17:54:13.252064 > > osd/PG.cc: 2619: FAILED assert(e.version > info.last_update) > > ceph version andisk-sprint-2-drop-3-390-g2dbd85c > (2dbd85c94cf27a1ff0419c5ea9359af7fe30e9b6) > > 1: (PG::add_log_entry(pg_log_entry_t&, ceph::buffer::list&)+0x481) > [0x733a61] > > 2: (PG::append_log(std::vector<pg_log_entry_t, > std::allocator<pg_log_entry_t> >&, eversion_t, > ObjectStore::Transaction&, bool)+0xdf) [0x74483f] > > 3: > (ReplicatedBackend::sub_op_modify(std::tr1::shared_ptr<OpRequest>)+0xcfe) > [0x8193be] > > 4: > (ReplicatedBackend::handle_message(std::tr1::shared_ptr<OpRequest>)+0x4a6) > [0x904586] > > 5: (ReplicatedPG::do_request(std::tr1::shared_ptr<OpRequest>, > ThreadPool::TPHandle&)+0x2db) [0x7aedcb] > > 6: (OSD::dequeue_op(boost::intrusive_ptr<PG>, > std::tr1::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x459) > [0x635719] > > 7: (OSD::ShardedOpWQ::_process(unsigned int, > ceph::heartbeat_handle_d*)+0x346) [0x635ce6] > > 8: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x8ce) > [0xa4a1ce] > > 9: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xa4c420] > > 10: (()+0x8182) [0x7f920f579182] > > 11: (clone()+0x6d) [0x7f920d91a30d] > > > > > > Raised tracker : http://tracker.ceph.com/issues/8887 > > > > Logs are attached to tracker. > > > > > > > > Thanks > > *Sahana Lokeshappa* > > * Test Development Engineer I *[image: cid:image001.png at 01CE9342.6D040E30] > 3rd Floor, Bagmane Laurel, Bagmane Tech Park > > C V Raman nagar, Bangalore 560093 > T: +918042422283 > > Sahana.Lokeshappa at SanDisk.com > > > > ------------------------------ > > PLEASE NOTE: The information contained in this electronic mail message is > intended only for the use of the designated recipient(s) named above. If > the reader of this message is not the intended recipient, you are hereby > notified that you have received this message in error and that any review, > dissemination, distribution, or copying of this message is strictly > prohibited. If you have received this communication in error, please notify > the sender by telephone or e-mail (as shown above) immediately and destroy > any and all copies of this message in your possession (whether hard copies > or electronically stored copies). > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140721/bd8cfa48/attachment.htm> -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.png Type: image/png Size: 3296 bytes Desc: not available URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140721/bd8cfa48/attachment.png>