>>We use Mellanox SX1036 and SX1012, which can function in 10 and 56GbE modes. It uses QSFP, Twinax or MPO, which terminates with LC fiber connections. While not dirt cheap, or entry >>level, we like these as being considerably cheaper than even a decent SDN solution. We have been able to build MLAG and leaf and spine solutions pretty easily with these. We use sx1012 too , pretty happy with them (in production since 2 years now). We are going to use new SN2100 (16 ports 40GB or 16 ports 100GB, with breakout cables that's 48 ports 10G or 48ports 25G). around 6000€ for the 40GB, and 12000€ with 100GB. (with mlx-os, but you can use also cumulus with theses new mellanox switches :) ----- Mail original ----- De: "Simon Leinen" <simon.leinen@xxxxxxxxx> À: "Erik McCormick" <emccormick@xxxxxxxxxxxxxxx> Cc: "ceph-users" <ceph-users@xxxxxxxxxxxxxx> Envoyé: Lundi 31 Octobre 2016 11:28:07 Objet: Re: 10Gbit switch advice for small ceph cluster upgrade Erik McCormick writes: > We use Edge-Core 5712-54x running Cumulus Linux. Anything off their > compatibility list would be good though. The switch is 48 10G sfp+ > ports. We just use copper cables with attached sfp. It also had 6 40G > ports. The switch cost around $4800 and the cumulus license is about > 3k for a perpetual license. Similar here, except we use Quanta switches (T5032-LY6). SFP+ slots and DAC cables. Actually our switches are 32*40GE, and we use "fan-out" DAC cables (QSFP on one side, 4 SFP+ on the other). Compared to 10GBaseT (RJ45), DAC cables are thicker, which may complicate cable management a little. On the other hand I think DAC still needs less power than 10GBaseT. And with the 40G setup, we have good port density and a smooth migration path to 40GE. We already use 40GE for our leaf-spine uplinks. Another advantage for us is that we can use a single SKU for both leaf and spine switches. The Cumulus licenses are a bit more expensive for those 40GE switches (as are the switches themselves), but it's still a good deal for us. Maybe these days it makes sense to look at 100GE switches in preference to 40GE; 100GE ports can normally be used as 2*50GE, 4*25GE, 1*40GE or 4*10GE as well, so the upgrade paths seem even nicer. And the prices are getting competitive I think. -- Simon. _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |