Hi Alexandre, thanks for that suggestion. mellanox might be on our shoping list already, but what regarding the redundandency design at all from your POV? /Götz Am 13.04.15 um 11:08 schrieb Alexandre DERUMIER: >>> So what would you suggest, what are your experiences? > > Hi, you can have a look at mellanox sx1012 for example > http://www.mellanox.com/page/products_dyn?product_family=163 > > 12 ports 40GB for around 4000€ > > you can use breakout cables to have 4x12 10GB ports. > > > They can be stacked with mlag and lacp > > > ----- Mail original ----- > De: "Götz Reinicke - IT Koordinator" <goetz.reinicke@xxxxxxxxxxxxxxx> > À: "ceph-users" <ceph-users@xxxxxxxxxxxxxx> > Envoyé: Lundi 13 Avril 2015 11:03:24 > Objet: Network redundancy pro and cons, best practice, suggestions? > > Dear ceph users, > > we are planing a ceph storage cluster from scratch. Might be up to 1 PB > within the next 3 years, multiple buildings, new network infrastructure > for the cluster etc. > > I had some excellent trainings on ceph, so the essential fundamentals > are familiar to me, and I know our goals/dreams can be reached. :) > > There is just "one tiny piece" in the design I'm currently unsure about :) > > Ceph follows some sort of keep it small and simple, e.g. dont use raid > controllers, use more boxes and disks, fast network etc. > > So from our current design we plan 40Gb Storage and Client LAN. > > Would you suggest to connect the OSD nodes redundant to both networks? > That would end up with 4 * 40Gb ports in each box, two Switches to > connect to. > > I'd think of OSD nodes with 12 - 16 * 4TB SATA disks for "high" io > pools. (+ currently SSD for journal, but may be until we start, levelDB, > rocksDB are ready ... ?) > > Later some less io bound pools for data archiving/backup. (bigger and > more Disks per node) > > We would also do some Cache tiering for some pools. > > From HP, Intel, Supermicron etc reference documentations, they use > usually non-redundant network connection. (single 10Gb) > > I know: redundancy keeps some headaches small, but also adds some more > complexity and increases the budget. (add network adapters, other > server, more switches, etc) > > So what would you suggest, what are your experiences? > > Thanks for any suggestion and feedback . Regards . Götz > -- Götz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 E-Mail goetz.reinicke@xxxxxxxxxxxxxxx Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: Jürgen Walter MdL Staatssekretär im Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg Geschäftsführer: Prof. Thomas Schadt
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
