Note: for significant fabric-related benefit, the *clients* need to be on the same fabric as the server nodes, not just the servers. We have a handful of servers, but hundreds of clients, so InfiniBand was out for us on economic grounds. And with just 1GbT (even with a high-end Cisco switch) we don't get the performance we desired on small files. If you have small files, many clients, and a small budget: beware. On 1/30/2012 11:05 AM, Peter Linder wrote: > Putting all machines on the same switch will at least save a > switch-hop, cutting latency by 1/3 in the ideal case which is an > improvement. If it will matter for your application, I do not know :) > > On 1/30/2012 6:45 PM, Dan Bretherton wrote: >> Thanks for the advice Peter, >>> You could get a cisco switch that supports cut through instead of >>> store-and-forward, for lower latency. >> Interesting option, but those cost ~?10K and are out of our price >> range unfortunately. The "cut through" switching technology is also >> available in Dell's new Force 10 range I believe. >> >>> Other than that, compare the port to port forwarding times and see if >>> there is a difference between the switches you are looking at (probably >>> not) and make your decision based on that. >> You're right, there isn't much of a difference between the forwarding >> rate of the 5548 and the 6248, both are about 100Mpps. They also >> have a similar bandwidth of about 180Gbps. However the 7048 does >> better on both measures, with forwarding rate of 160Mpps and a >> bandwidth of 224Gbps. Unfortunately the 7048 costs five times as >> much as the 5548, and I don't know if the users would notice any >> difference at all. I expect some would and some wouldn't. >> >> -Dan. >> >> On 01/27/2012 01:48 PM, gluster-users-request at gluster.org wrote: >>> Date: Fri, 27 Jan 2012 14:16:36 +0100 >>> From: Peter Linder<peter.linder at fiberdirekt.se> >>> Subject: Re: Switch recommendations >>> To:gluster-users at gluster.org >>> Message-ID:<4F22A3B4.8000008 at fiberdirekt.se> >>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>> >>> You could get a cisco switch that supports cut through instead of >>> store-and-forward, for lower latency. >>> >>> Other than that, compare the port to port forwarding times and see if >>> there is a difference between the switches you are looking at (probably >>> not) and make your decision based on that. Consider connecting >>> everything to two switches, for failover in case a switch breaks? >>> >>> On 1/27/2012 2:04 PM, Dan Bretherton wrote: >>>> > Dear All, >>>> > I need to buy a bigger GigE switch for my GlusterFS cluster and >>>> I am >>>> > trying to decide whether or not a much more expensive one would be >>>> > justified. I have limited experience with networking so I don't >>>> know >>>> > if it would be appropriate to spend ?500, ?1500 or ?3500 for a >>>> 48-port >>>> > switch. Those rough costs are based on a comparison of 3 Dell >>>> > Powerconnect switches: the 5548 (bigger version of what we have >>>> now), >>>> > the 6248 and the 7048. The servers in the cluster are nothing >>>> special >>>> > - mostly Supermicro with SATA drives and 1GigE network adapters. I >>>> > can only justify spending more than ~?500 if I can be sure that >>>> users >>>> > would notice the difference. Some of the users' applications do >>>> lots >>>> > of small reads and writes, and they do run much more slowly if >>>> all the >>>> > servers are not connected to the same switch, as is the case now >>>> while >>>> > I don't have a big enough switch. Any advice or comments would be >>>> > much appreciated. >>>> > >>>> > Regards >>>> > Dan. > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
