On Sat, Jan 28, 2012 at 05:31:28PM -0600, D. Dante Lorenso wrote: > Thinking about buying 8 servers with 4 x 2TB 7200 rpm SATA drives > (expandable to 8 drives). Each server will have 8 network ports and > will be connected to a SAN switch using 4 ports link aggregated and > connected to a LAN switch using the other 4 ports aggregated. The > servers will run CentOS 6.2 Linux. The LAN side will run Samba and > export the network shares, and the SAN side will run Gluster daemon. Just a terminology issue, but Gluster isn't really a SAN, it's a distributed NAS. A SAN uses a block-level protocol (e.g. iSCSI), into which the client runs a regular filesystem like ext4 or xfs or whatever. A NAS is a file-sharing protocol (e.g. NFS). Gluster is the latter. > With 8 machines and 4 ports for SAN each, I need 32 ports total. > I'm thinking a 48 port switch would work well as a SAN back-end > switch giving me left over space to add iSCSI devices and backup > servers which need to hook into the SAN. Out of interest, why are you considering two different network fabrics? Are there one set of clients which are talking CIFS and a different set of clients using the Gluster native client? > 4) Performance tuning. So far, I've discovered using dd and iperf > to debug my transfer rates. I use dd to test raw speed of the > underlying disks (should I use RAID 0, RAID 1, RAID 5 ?) Try some dd measurements onto a RAID 5 volume, especially for writing, and you'll find it sucks. I also suggest something like bonnie++ to get a more realistic performance measurement than just the dd throughput, as it will include seeks and filesystem operations (e.g. file creations/deletions) > Perhaps if my drives on each of the 8 > servers are RAID 0, then I can use "replicate 2" through gluster and > get the "RAID 1" equivalent. I think using replicate 2 in gluster > will 1/2 my network write/read speed, though. In theory Gluster replication ought to improve your read speed, since some clients can access one copy spindle while other clients access the other. But I'm not sure how much it will impact the write speed. I would however suggest that building a local RAID 0 array is probably a bad idea, because if one disk of the set fails, that whole filesystem is toast. Gluster does give you the option of a "distributed replicated" volume, so you can get both the "RAID 0" and "RAID 1" functionality. HTH, Brian.
