Solving my own issue here. gluster can be frustrating at times. Documentation, bug tracker and mailing list all say that various replace-brick functions are deprecated in 3.6 and self-heal daemon is responsible for migrating data. the CLI even says tell you when you try any command: All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) Yet it seems replace-brick still works, and is infact the only way to migrate data from one brick to another as SHD clearly does not work as per my previous message. Round 2 - aka ignoring everything everybody says and doing it the old way $ gluster volume info test Volume Name: test Type: Replicate Volume ID: 34716d2f-8fd1-40a4-bcab-ecfe3f6d116b Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: ds1:/export/test Brick2: ds2:/export/test $ gluster volume replace-brick test ds2:/export/test ds2:/export/test2 start force All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) y volume replace-brick: success: replace-brick started successfully ID: 6f8054cc-03b8-4a1e-afc5-2aecf70d909b $ gluster volume replace-brick test ds2:/export/test ds2:/export/test2 status All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) y volume replace-brick: success: Number of files migrated = 161 Migration complete $ gluster volume replace-brick test ds2:/export/test ds2:/export/test2 commit All replace-brick commands except commit force are deprecated. Do you want to continue? (y/n) y volume replace-brick: success: replace-brick commit successful $ gluster volume info test Volume Name: test Type: Replicate Volume ID: 34716d2f-8fd1-40a4-bcab-ecfe3f6d116b Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: ds1:/export/test Brick2: ds2:/export/test2 fuse clients can now see all data as expected. I can't begin to express how frustrating this experience has been. On 06/02/15 10:51, Jordan Tomkinson wrote: > Hi, > > Using Gluster 3.6.1, I'm trying to replace a brick but after issuing a > volume heal nothing gets healed and my clients see an empty volume. > > I have reproduced this on a test volume, shown here. > > $ gluster volume status test > > Status of volume: test > Gluster process Port Online Pid > ------------------------------------------------------------------------------ > Brick ds1:/export/test 49153 Y 7093 > Brick ds2:/export/test 49154 Y 11472 > NFS Server on localhost 2049 Y 11484 > Self-heal Daemon on localhost N/A Y 11491 > NFS Server on 10.42.0.207 2049 Y 7110 > Self-heal Daemon on 10.42.0.207 N/A Y 7117 > > Task Status of Volume test > ------------------------------------------------------------------------------ > There are no active volume tasks > > I then mount the volume from a client and store some files. > > Now I replace ds2:/export/test with an empty disk mounted on > ds2:/export/test2 > > $ gluster volume replace-brick test ds2:/export/test ds2:/export/test2 > commit force > volume replace-brick: success: replace-brick commit successful > > At this point, doing an ls on the volume mounted from a fuse client > shows an empty volume, basically the contents of the new empty brick. > > So i issue a volume heal full > > $ gluster volume heal test full > Launching heal operation to perform full self heal on volume test has > been successful > Use heal info commands to check status > > $ gluster volume heal test info > Gathering list of entries to be healed on volume test has been successful > > Brick ds1:/export/test > Number of entries: 0 > > Brick ds2:/export/test2 > Number of entries: 0 > > > Nothing gets healed from ds1:/export/test to ds2:/export/test2 and my > clients still see an empty volume. > > I can see the data on ds1:/export/test if i look inside the brick > directory, but nothing on ds2:/export/test2 > > tailing glustershd.log, nothing is printed after running the heal command. > > the only log is one line from etc-glusterfs-glusterd.vol.log > [2015-02-06 02:48:39.906224] I > [glusterd-volume-ops.c:482:__glusterd_handle_cli_heal_volume] > 0-management: Received heal vol req for volume test > > Any ideas? > > > -- Jordan Tomkinson System Administrator Catalyst IT Australia _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
