Mike - Gluster does not use LVM, but we do support using it under Gluster. There are only three use cases for Gluster and LVM that make sense. 1. You want to snapshot the filesystem (logical volume). 2. Your RAID controller won't create multiple LUN's and you have to export a very large filesystem. You can use LVM to carve that filesystem into multiple, smaller logical volumes for faster fscks and (see 3). 3. You have a Gluster Storage server with more disk space that others. Gluster will perform better if every brick in the cluster is the same size. If you are using a brick size of 2TB and you add a server with 10TB to the cluster you should, for best performance, create 5 2TB logical volumes with LVM then add all of those bricks to a volume; gluster> volume add-brick avolume server1:/dev/vg1/lv1 gluster> volume add-brick avolume server1:/dev/vg1/lv2 gluster> volume add-brick avolume server1:/dev/vg1/lv3 gluster> volume add-brick avolume server1:/dev/vg1/lv4 gluster> volume add-brick avolume server1:/dev/vg1/lv5 and of course gluster> volume rebalance avolume start Remember, your smallest brick size MUST be larger that your largest file. Once your Gluster volumes are created Gluster will hide all of the bricks under a single namespace. Thanks, Craig --> Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.carl at gmail.com From: "Mike Hanby" <mhanby at uab.edu> To: gluster-users at gluster.org Sent: Monday, October 25, 2010 3:45:29 PM Subject: Re: Question about Volume Type when bricks are on SAN So based on the suggestions from both you and Daniel many smaller LUNs should be better for performance and maintenance. Question regarding LVM, do you use LVM to place all of the luns into a single volume group and ultimately a single logical volume / brick? Or does each LUN become it's own volume group and logical volume and brick? Based on what you suggested regarding fsck on smaller file systems I suspect the latter, where you'd have many smaller bricks per server rather than a single large brick comprised of many luns in a single logical volume. Thanks, Mike -----Original Message----- From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] On Behalf Of Craig Carl Sent: Sunday, October 24, 2010 2:15 AM To: Daniel Mons Cc: gluster-users at gluster.org Subject: Re: Question about Volume Type when bricks are on SAN Another good reason to keep your LUN's small is the possibility of having to fsck the filesystem. Even on really fast disk it can take days to fsck a 16TB LUN. Gluster hides the multiple LUNs in a volume so you never have to deal with them, other than a bit more SAN management and more typing to setup a volume the first time there is no downside. Thanks, Craig --> Craig Carl Senior Systems Engineer Gluster From: "Daniel Mons" <daemons at kanuka.com.au> To: gluster-users at gluster.org Sent: Saturday, October 23, 2010 5:38:12 AM Subject: Re: Question about Volume Type when bricks are on SAN On Sat, Oct 23, 2010 at 5:22 AM, Mike Hanby <mhanby at uab.edu> wrote: > Each Gluster server has been allocated a single 9TB LUN (if it makes any sense to do so, I could have the SAN admin provide many smaller LUNs per server and use LVM to combine into single brick) Generally speaking you should LVM over many smaller LUNs. With multiport FC HBAs being the standard, you will get better performance as separate requests hit separate LUNs. Multiple LUNs mean multiple SCSI queues, which can reduce I/O latency substantially. We try to cap our LUNs at around 100GB. It does make for a lot of LUNs per server, but it's worth if for the performance gain. Talk to your storage admin and ask them what their recommendation is based on your vendor and their best practices. -Dan _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
