Hi, During deep-scrub Ceph discovered some inconsistency between OSDs on my cluster (size 3, min size 2). I have fund broken object and calculated md5sum of it on each OSD (osd.195 is acting_primary): osd.195 - md5sum_aaaa osd.40 - md5sum_aaaa osd.314 - md5sum_bbbb I run ceph pg repair and Ceph successfully reported that everything went OK. I checked md5sum of the objects again: osd.195 - md5sum_bbbb osd.40 - md5sum_bbbb osd.314 - md5sum_bbbb This is a bit odd. How Ceph decides which copy is the correct one? Based on last modification time/sequence number (or similar)? If yes, then why object can be stored on one node only? If not, then why Ceph selected osd.314 as a correct one? What would happen if osd.314 goes down? Will ceph return wrong (old?) data, even with three copies and no failure in the cluster? For now I'm unable to reproduce this on my test cluster. I'll post here if I reproduce this. Thanks for any help, PS _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |