On Wed, Feb 15, 2012 at 08:22:18PM -0500, Miles Fidelman wrote: > We've been running a 2-node, high-availability cluster - basically xen > w/ pacemaker and DRBD for replicating disks. We recently purchased 2 > additional, servers, and I'm thinking about combining all 4 machines > into a 4-node cluster - which takes us out of DRBD space and requires > some other kind of filesystem replication. > > Gluster, Ceph, Sheepdog, and XtreemFS seem to keep coming up as things > that might work, but... Sheepdog is too tied to KVM ... although if you're considering changing DRBD->Gluster, then changing Xen->KVM is perhaps worth considering too? Having VM images as single files in a filesystem arguably allows you to back them up more easily, and if there are hotspots you can identify them or shift load around manually. But I do like the idea of the fully distributed block device provided by sheepdog. > i. Is it now reasonable to consider running Gluster and Xen on the same > boxes, without hitting too much of a performance penalty? I have been testing Gluster on 24-disk nodes : - 2 HBAs per node (one 16-port and one 8-port) - single CPU chip (one node is dual-core i3, one is quad-core Xeon) - 8GB RAM - 10G ethernet and however I hit it, the CPU is mostly idle. I think the issue for you is more likely to be one of latency rather than throughput or CPU utilisation, and if you have multiple VMs accessing the disk concurrently then latency becomes less important. However, I should add that I'm not running VMs on top of this, just doing filesystem tests (and mostly reads at this stage). For what gluster 3.3 will bring to the table, see this: http://community.gluster.org/q/can-i-use-glusterfs-as-an-alternative-network-storage-backing-for-vm-hosting/ Regards, Brian.
