Does anyone know if one can mix infiniband and tcp/ip ? Thanks On Fri, Jan 14, 2011 at 11:16 AM, Di Pe <dipeit at gmail.com> wrote: > Hi, > > we would like to build a gluster storage systems that combines our > need for performance with our need for disaster recovery. I saw a > couple of posts indicating that this is possible > (http://gluster.org/pipermail/gluster-users/2010-February/003862.html) > but am not 100% clear if that is possible > > Let's assume I have a total of 6 storage servers and bricks and want > to spread them across 2 buildings for DR. server1-3 are in building1 > server 4-6 are in building2. I create a distributed replicated volume > and make sure that server1 replicates to server4, server2 to server5 > and server3 to server6. I have this running today and it's working > reasonably well. Now I have a couple of high performance compute > systems in the server room of building 1 are currently using glusterfs > to talk to the gluster storage cluster. Throughput is great, latency > not so. I believe I have 2 options: > > 1. connecting server1,2,3 to my new IB switch (with rdma) and keep my > compute servers connected via a tcp mount point /mnt/gluster. Latency > will be reduced during normal operations. Gluster will figure out to > use server1,2,3 because they can serve the files much quicker and will > leave server4-6 for replication. If a server2 dies server4 will > automatically take over serving files. Performance is reduced for many > files but the end user does not experience and outage. Is this > correct? > > 2. connecting server1,2,3 AND my compute systems to my new IB switch > and mount storage via an rdma mount point /mnt/gluster-fast. ?End > users will experience even lower latency. If server2 dies users will > continue to be able to access files that were residing on the other 2 > servers, however all files that were on server2 become unavailable and > long running compute jobs might die. The end user needs to switch to > tcp mount point /mnt/gluster to be able to access the files on server2 > and other servers. Is this correct? > > > Are my assumptions correct? > > Should I use 2 mount points to /mnt/gluster and /mnt/gluster-fast to > give users the choice between very reliable and very performant? > > Thanks > dipe >
