Just to follow-up in case anyone else goes through this procedure, I was successful in performing the migration based on the steps above. After doing the hard drive swap I am currently running a rebalance after running fix-layout. Louis --- Louis R. Marascio 512-964-4569 On Fri, Jul 17, 2015 at 1:21 PM, Louis Marascio <marascio@xxxxxxxxx> wrote: > I currently have a 30 TB distributed-replicate gluster volume with 10 bricks > and a replica count of 2 (so 5x2 configuration). The cluster is made up of 5 > nodes, and each node has 4 3.5" HD slots. Each node has 2 bricks and each > brick is comprised of 2 3 TB hard drives in a raid 0 configuration. As you > can see, each brick is 6 TB. > > The volume is getting quite full and I need to stretch the hardware as long as > possible, so my plan is to migrate away from bricks build around 2 3 TB drives > in raid 0 to a single 6 TB drive. By doing this I will free up 2 HD slots per > node and can then add new bricks doubling the capacity of the volume to 60 TB. > > To the crux of the matter: I'm looking for any advice or thoughts on doing > this migration. After some discussion on IRC shyam and I came to the > conclussion that the following procedure was a good option (downtime is > ok): > > 1. Run a heal operation on the entire volume. > > 2. Shutdown all cluster nodes. > > 3. Pull sda and sdb (slot1 and slot2) from the node to upgrade. In my > config sd[ab] = md1 = brick1. Due to how my replication is setup I will > be doing the hard drive swap starting with brick2. > > 4. Insert a new 6 TB disk into slot1 (where sda was). > > 5. Boot the node from a livecd or equivalent. We should have a 6 TB drive > as sda and brick2 should be md2 (comprising sdc and sdd). > > 6. Clone /dev/md2 onto /dev/sda. I will do this with either dd or cat. > Suffice it to say I will create a bit-perfect copy of /dev/md2 onto > /dev/sda thus cloning all of brick2's data. > > 7. Zero out the mdadm super block on /dev/sda (I believe the superblock is > going to be picked up as part of the clone and we don't need it > anymore). > > 8. My fstab uses labels for mount points so I will ensure the label of the > new 6 TB sda drive is the same as the old md1 (just cloned, so it SHOULD > be right). > > 9. Shutdown the node. > > 10. Pull sdc and sdb (slot3 and slot4). Move newly cloned 6 TB drive from > slot1 to slot3 (where sdc was). Replace original sda and sdb into their > original slots (remember, these are for brick1 and not being modified at > all at this point). I should now have brick1 back, brick2 should exist > on a single 6 TB hard drive, and there should be a single free slot for > our new brick, brick3, to be added later after ensuring we have migrated > brick2 successfuly. > > 11. Boot the nodes. Gluster should see no change as the 6 TB drive is > mounted at the same spot as the previous md1 brick was. > > At this point adding the new 6 TB brick into the newly freed slot is > straightforward enough. Since I have a replica 2 setup I will obviously do the > above steps on two nodes at a time so that the new 6 TB brick has somewhere to > replicate to. > > Can anyone see any problems with this approach? Is there a better way? > > Thanks (especially to shyam for helping out on IRC)! > > Louis > > note: the bricks are actually 5.4TB each and the volume is actually 27 TB as > a small bit of each drive is carved out for the boot partition. I omit these > numbers above to keep things simple and the numbers round. > > --- > Louis R. Marascio > 512-964-4569 _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
