hi,
In releases till now from day-1 with replication, there is a
corner case bug which can wipe out the all the bricks in that replica
set when the disk/brick(s) are replaced.
Here are the steps that could lead to that situation:
0) Clients are operating on the volume and are actively pumping data.
1) Execute replace-brick command (OR) take down the brick that needs the
disk replaced or re-formatted. and bring the brick back up.
2) Client creates a file/directory just on the root of the brick which
succeeds on the new brick but fails on the bricks that have been online
(May be because the file existed on the bricks that are good copies)
3) Now when self-heal is triggered from the Client/self-heal-daemon it
thinks the just replaced brick is the correct directory and deletes the
files/directories from the bricks that have the actual data.
I have been working on afr for almost 4 years now and never saw any user
complain about this problem. We were working on a document for an
official way to replace brick/disk but it never occurred to us that
this could happen until recently. I am going to get a proper document
by end of this week on replacing the bricks/disks in a safe way. And
will keep you guys posted about fixes to prevent this from happening
entirely.
Pranith
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
