Hello, firstly, I'm not the main network guy here by a long shot, OTOH I do know a thing or two, my they just be from trial and error. On Wed, 1 Jun 2016 09:49:53 +0200 David Riedl wrote: > Hello everybody, > > we want to upgrade/fix our SAN switches. I kinda screwed up when I was > first planning our CEPH storage cluster. > > Right now we have 2 x HP 2530-24G Switch (J9776A). We have 3 server each > outfittet with 2 x 4 gigabit cards. (Don't judge me, I also was on a > budget) > So 3 servers are the entirety of your Ceph storage nodes, right? > Each card goes with 2 cables to one of the two switches for redundancy. > > The bonds on the servers are configured as mode 0 (load balancing > (round-robin)). > > The cluster works but I guess due to high drop rate on the switch > interfaces we have a pretty bad latency. > Have you been able to determine what causes the drops? My first guess would be that this bonding is simply not compatible with what the switches can do/expect. > I also read that LACP would be the answer to my problem, since I want to > utilize all interfaces and have redundancy at the same time. > LACP isn't round-robin, but it does distribute things in fashion and given the fact that it actually works you should try it. To be more specific, LACP distribution is based on "sessions", so if you have enough variety in there you will get something that's good enough. A single session however will not be faster than an individual link, IIRC. > > So, back to my Question: > > What 48 port gigabit switch is the best replacement for that type of > configuration? 10GB is not an option. > Why a single switch and thus a SPoF? Or are you planning to get 2 switches and plan for more clients and Ceph nodes down the road? If I were in your shoes, I'd look at 2 switches running MC-LAG (in any of the happy variations there are) https://en.wikipedia.org/wiki/MC-LAG And since you're on a budget, something like the Cumulus based offerings (Penguin computing, etc). Christian > > > > Regards > > David > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Rakuten Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |