The CRS3xx series is no-frills but works. I have a 6-node cluster running just fine on them. Bonding should work also for 2x10G. Interestingly enough, MLAG was introduced in today's routeros v7 beta, but I'm skeptical that it will be stable yet. On Wed, May 19, 2021 at 1:22 AM Hermann Himmelbauer <hermann@xxxxxxx> wrote: > Dear Ceph users, > I am currently constructing a small hyperconverged Proxmox cluster with > ceph as storage. So far I always had 3 nodes, which I directly linked > together via 2 bonded 10G network interfaces for the Ceph storage, so I > never needed any switching devices. > > This new cluster has more nodes, so I am considering using a 10G switch > for the storage network. As I have no experience with such a setup, I > wonder if there are any specific issues that I should think of > (latency...)? > > As the whole cluster should be not too expensive, I am currently > thinking of the following solution: > > 2* CRS317-1G-16s+RM switches: > https://mikrotik.com/product/crs317_1g_16s_rm#fndtn-testresults > > SFP+ Cables like these: > https://www.fs.com/de/products/48883.html > > Some network interface for each node with two SFP+ ports, e.g.: > > https://ark.intel.com/content/www/de/de/ark/products/39776/intel-ethernet-converged-network-adapter-x520-da2.html > > Connect each port with each switch and configure master/slave > configuration so that the switches are redundant. > > What do you think of this setup - or is there any information / > recommendation for an optimized setup of a 10G storage network? > > Best Regards, > Hermann > > -- > hermann@xxxxxxx > PGP/GPG: 299893C7 (on keyservers) > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > -- Jeremy Austin jhaustin@xxxxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |