Initial setup: A replicated volume with 3 bricks Goal: Remove one of the bricks from it. Version: # glusterfs 3.3git built on Jun 7 2013 14:38:02 (branch release-3.3) Initial setup: A replicated volume with 3 bricks Goal: Remove one of the bricks from it. Outcome: A completely broken volume ------------- Volume info ------------- root at fs-14.example:~# gluster volume info Volume Name: test-fs-cluster-1 Type: Replicate Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7 Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: fs-14.example.com:/mnt/brick21 Brick2: fs-15.example.com:/mnt/brick20 Brick3: fs-14.example.com:/mnt/brick33 ------------- Trying to remove a brick ------------- fields-config-gluster.rb[5035]: Using commandline: gluster volume remove-brick test-fs-cluster-1 replica 2 fs-14.example.com:/mnt/brick33 start fields-config-gluster.rb[5035]: Command returned exit code 255: gluster volume remove-brick test-fs-cluster-1 replica 2 fs-14.example.com:/mnt/brick33 start stdout was: stderr was: Remove Brick start unsuccessful ------------- Volume turned Distributed-Replicate ------------- [12:23:37] root at fs-14.example:~# gluster volume info Volume Name: test-fs-cluster-1 Type: Distributed-Replicate Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7 Status: Started Number of Bricks: 1 x 2 = 3 Transport-type: tcp Bricks: Brick1: fs-14.example.com:/mnt/brick21 Brick2: fs-15.example.com:/mnt/brick20 Brick3: fs-14.example.com:/mnt/brick33 ------------- Trying to remove brick again ------------- [12:26:20] root at fs-14.example:~# gluster volume remove-brick test-fs-cluster-1 replica 2 fs-14.example.com:/mnt/brick33 start number of bricks provided (1) is not valid. need at least 2 (or 2xN) ------------- Trying to stop volume ------------- [12:28:34] root at fs-14.example:~# gluster volume stop test-fs-cluster-1 Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y Stopping volume test-fs-cluster-1 has been successful ------------- Trying to start volume again ------------- [12:29:03] root at fs-14.example:~# gluster volume start test-fs-cluster-1 Starting volume test-fs-cluster-1 has been unsuccessful ------------- Trying to stop volume again ------------- [12:29:49] root at fs-14.example:~# gluster volume stop test-fs-cluster-1 Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y Volume test-fs-cluster-1 is not in the started state ------------- Trying to delete volume ------------- [12:29:55] root at fs-14.example:~# gluster volume delete test-fs-cluster-1 Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y Volume test-fs-cluster-1 has been started.Volume needs to be stopped before deletion. ------------- Checking volume info ------------- # gluster volume info Volume Name: test-fs-cluster-1 Type: Distributed-Replicate Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7 Status: Started Number of Bricks: 1 x 2 = 3 Transport-type: tcp Bricks: Brick1: fs-14.example.com:/mnt/brick21 Brick2: fs-15.example.com:/mnt/brick20 Brick3: fs-14.example.com:/mnt/brick33 ------------- Trying to stop volume again ------------- [12:30:50] root at fs-14.example:~# gluster volume stop test-fs-cluster-1 Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y Volume test-fs-cluster-1 is not in the started state ------------- Restarting glusterfs-server ------------- [12:38:05] root at fs-14.example:~# /etc/init.d/glusterfs-server restart glusterfs-server start/running, process 6426 ------------- Volume switched back to "Replicate" ------------- [12:38:33] root at fs-14.example:~# gluster volume info Volume Name: test-fs-cluster-1 Type: Replicate Volume ID: 752e7ffd-04bb-4234-8d16-d1f49ef510b7 Status: Started Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: fs-14.example.com:/mnt/brick21 Brick2: fs-15.example.com:/mnt/brick20 Brick3: fs-14.example.com:/mnt/brick33 ------------- Trying to stop volume again ------------- [12:38:39] root at fs-14.example:~# gluster volume stop test-fs-cluster-1 Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y Volume test-fs-cluster-1 is not in the started state Any idea what's up with that? Cheers, Marc