How to replace a dead brick? (3.6.5)

[Lindsay Mathieson <lindsay.mathieson@xxxxxxxxx>]

First up - one of the things that concerns me re gluster is the incoherent state of documentation. The only docs linked on the main webpage are for 3.2 and there is almost nothing on how to handle failure modes such as dead disks/bricks etc, which is one of glusters primary functions.

My problem - I have a replica 2 volume, 2 nodes, 2 bricks (zfs datasets).

As a test, I destroyed one brick (zfs destroy the dataset).

Can't start the datastore1:

  volume start: datastore1: failed: Failed to find brick directory /glusterdata/datastore1 for volume datastore1. Reason : No such file or directory

A bit disturbing, I was hoping it would work off the remaining brick.

Can't replace the brick:

  gluster volume replace-brick datastore1 vnb.proxmox.softlog:/glusterdata/datastore1 vnb.proxmox.softlog:/glusterdata/datastore1-2 commit force

because the store is not running.

After a lot of googling I found list messages referencing the remove brick command:

gluster volume remove-brick datastore1 replica 2 vnb.proxmox.softlog:/glusterdata/datastore1c commit force

Fails with the unhelpful error:

wrong brick type: commit, use <HOSTNAME>:<export-dir-abs-path>
Usage: volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ... <start|stop|status|commit|force>

In the end I destroyed and recreated the volume so I could resume testing, but I have no idea how I would handle a real failed brick in the future

--

Lindsay

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users