Hi Josh: there is one object become *unfound* in my cluster, i think it is relate to the feature "recovery_deletes",so first i want to kown how to disable this feature. i found one shard(not primary) of ec object left a 'pg_missing_item' in the pg's meta and a 'may_include_deletes_in_missing' 'missing/0000000000000019.5EB6CDE7.head.rbd%udata%e26%eae022c74b0dc51%e000000000247d221..' when doing peering , the pg wanna to recovery the object, but other shards declare that the oid meta did not in db, so i think it maybe deleted when recovering... 2018-04-23 17:55:16.894148 7f4def0de700 20 osd.196 pg_epoch: 19081 pg[25.be5s0( v 19079'27507 (17510'23800,19079'27507] local-lis/les=19080/19081 n=17360 ec=17077/17077 lis/c 19080/17682 les/c/f 19081/17689/0 19080/19080/19080) [196,103,223,27,76,142] r=0 lpr=19080 pi=[17682,19080)/5 rops=1 crt=19079'27507 mlcod 0'0 active+recovering+degraded] handle_sub_read_reply shard=103(1) error=-2 2018-04-23 17:55:16.894634 7f4def0de700 20 osd.196 pg_epoch: 19081 pg[25.be5s0( v 19079'27507 (17510'23800,19079'27507] local-lis/les=19080/19081 n=17360 ec=17077/17077 lis/c 19080/17682 les/c/f 19081/17689/0 19080/19080/19080) [196,103,223,27,76,142] r=0 lpr=19080 pi=[17682,19080)/5 rops=1 crt=19079'27507 mlcod 0'0 active+recovering+degraded] handle_sub_read_reply shard=27(3) error=-2 2018-04-23 17:55:16.895423 7f4def0de700 20 osd.196 pg_epoch: 19081 pg[25.be5s0( v 19079'27507 (17510'23800,19079'27507] local-lis/les=19080/19081 n=17360 ec=17077/17077 lis/c 19080/17682 les/c/f 19081/17689/0 19080/19080/19080) [196,103,223,27,76,142] r=0 lpr=19080 pi=[17682,19080)/5 rops=1 crt=19079'27507 mlcod 0'0 active+recovering+degraded] handle_sub_read_reply shard=76(4) error=-2 best regards -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html