On Thu, Jun 28, 2012 at 10:40:43AM -0500, Nathan Stratton wrote: > But wait, yes, I have 16 physical disks, but I am running distribute > + replicate so the 8 physical boxes are broken up into 4 pairs of > redundant boxes. When I do a write, I am writing on two servers, or > 4 physical disks. So in my case, 31.1 MB/s vs about 200 MB/s native > is not that bad. > > DRDB is MUCH faster, but your not comparing apples to apples. DRBD > has worked great for me in the past when I only needed two storage > nodes to be mirrored in active/active, but as soon as you grow past > that you need to look at something like Gluster. But we're talking different things here: * VM image (i.e. the root filesystem it boots from; where the O/S sits; logs and scratch space) * Application data storage You'd be mad to have terabytes of data sitting inside a single VM image file. It's unshareable, the VM image is one big humungous blob, and to back it up effectively you need to run the backup tools within the VM itself. Furthermore, the performance of GlusterFS is excellent when you mount it directly. It only sucks when you're using a gluster-mounted file as a KVM virtual disk. So what I'm suggesting is, if you need performance today: * Use DRBD+LVM for your VM filesystem storage * Use glusterfs for your "big data", and attach it to those VM(s) which need to access it - leveraging the naturally shared nature of glusterfs. And eventually you'll be able to simplify your system by migrating your VM images to glusterfs, when performance catches up. Ganeti can manage both types of cluster, so you don't lose out by learning it up-front. Regards, Brian.
