Thank you for your explanation. > Every time 2 of 18 OSDs are crashing. I think it's happening when run PG replication because crashing only 2 OSDs and every time they're are the same. 1st you said, 2 osds were crashed every time. From the log you pasted, it makes sense to do something for osd.3. > rm -rf > /var/lib/ceph/osd/ceph-4/current/3.2_head/rb.0.19f2e.238e1f29.000000000728__head_813E90A3__3 What makes me confused now is this. Was osd.4 also crashed like osd.3? > -1> 2016-02-24 04:51:45.904673 7fd995026700 5 -- op tracker -- , seq: 19231, time: 2016-02-24 04:51:45.904673, event: started, request: osd_op(osd.13.12097:806247 rb.0.218d6.238e1f29.000000010db3 [copy-get max 8388608] 3.94c2bed2 ack+read+ignore_cache+ignore_overlay+map_snap_clone e13252) v4 And crash seems to happen during this process, what I really want to know is what this message inferred. Did you check osd.13? Anyhow your cluster is now fine...no? That's good news. Cheers, Shinobu On Fri, Mar 4, 2016 at 11:05 AM, Alexander Gubanov <shtnik@xxxxxxxxx> wrote: > I decided to refuse use of ssd cache pool and create just 2 pool. 1st pool > only of ssd for fast storage 2nd only of hdd for slow storage. > What about this file, honestly, I don't know why it is created. As I say I > flush the journal for fallen OSD and remove this file and then I start osd > damon: > > ceph-osd --flush-journal osd.3 > rm -rf > /var/lib/ceph/osd/ceph-4/current/3.2_head/rb.0.19f2e.238e1f29.000000000728__head_813E90A3__3 > service ceph start osd.3 > > But if I turn the cache pool off the file isn't created: > > ceph osd tier cache-mode ${cahec_pool} forward > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Email: shinobu@xxxxxxxxx GitHub: shinobu-x Blog: Life with Distributed Computational System based on OpenSource _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com