On Thu, Oct 8, 2015 at 6:45 PM, Francois Lafont <flafdivers@xxxxxxx> wrote: > Hi, > > On 08/10/2015 22:25, Gregory Farnum wrote: > >> So that means there's no automated way to guarantee the right copy of >> an object when scrubbing. If you have 3+ copies I'd recommend checking >> each of them and picking the one that's duplicated... > > It's curious because I have already tried with cephfs to "corrupt" a > file in the OSD backend. I had a little text file in cephfs mapped to > the object "$inode.$num" and this object was in the PG $pg_id, in the > primary OSD $primary and in the secondary OSD $secondary (I had indeed > size == 2). I thought that the primary OSD was always taken as reference > by the "ceph pg repair" command, so I have tried this: > > # Test A > echo "foo blabla..." >/var/lib/ceph/osd/ceph-$primary/current/$pg_id_head/$inode.$num > ceph pg repair $pg_id > > and the "repair" command worked correctly and my file was repaired > correctly. I have tried to change the file in the secondary OSD too with: > > # Test B > echo "foo blabla..." >/var/lib/ceph/osd/ceph-$secondary/current/$pg_id_head/$inode.$num > ceph pg repair $pg_id > > and it was the same, the file was repaired correctly too. In these 2 > cases, the good OSD was taken as reference (the secondary for the test > A and the primary for the test B). > > So, in this case, how did ceph know which copy was the correct object? The size of the on-disk file didn't match the OSD's record of the object size, so it rejected it. This works for that kind of gross change, but it won't catch stuff like a partial overwrite or loss of data within a file. -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |