> -----Original Message----- > From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of > Gandalf Corvotempesta > Sent: 15 June 2016 17:03 > To: ceph-users@xxxxxxxx > Subject: Switches and latency > > Let's assume a fully redundant network. > We need 4 switches, 2 for the public network, 2 for the cluster network. I would reconsider if you need separate switches for each network, vlans would normally be sufficient. If bandwidth is not an issue, you could even tag both vlans over the same uplinks. Then there is the discussion around whether separate networks are really essential.... > > 10GBase-T has higher latency than SFP+ but are also cheaper, as manu new > servers ha 10GBaseT integrated onboard and there is no need for twinax > cables or transceaver. Very true and is one for the reasons we switched to it, the other being I was fed up having to solve the "this cable doesn't work with this switch/NIC" challenge. Why cables need eeprom to say which devices they will work with is lost on me!!! On the latency front I wouldn't be too concerned. 10GB-T has about 2us more latency per hop than SFP. Lowest latency's commonly seen in Ceph are around 500-1000us for reads and 2000us for writes. So unless you are trying to get every last 0.01 of a percent, I don't think you will notice. It might be wise to link the switches together though with SFP or 40G, so the higher latency only effects the last hop to the host and will put you in a better place if/when you need to scale your network out. > > I think that low latency is needed on the public networks, where servers has > to read data from and not on the cluster network, used only for replication > purposes. > > What do you think ? using SFP+ everywhere would increase the total cost. > And what about Infiniband (IPoIB) for the cluster network and SFP+ for the > public network? IB refurbished switches are cheaper than 10GB SFP+ > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |