|
God morning, * 2 x SN2100 100Gb/s Switch 16 portsWhich incidentally is a half sized (identical HW really) Arctica 3200C. really never heart from them :-) (and didn't find any price €/$ region)
[...]* 10 x ConnectX 4LX-EN 25Gb card for hypervisor and OSD nodes You haven't commented on my rather lengthy mail about your whole design, maybe accidentally skipped, so much new input :-) sorry
The above will give you a beautiful, fast (but I doubt you'll need the Jepp, they do MLAG (with the 25Gbit version of the cx4 NICs)
In more technical terms, your network as depicted above can handle under before we started with planing Ceph / new hypervisor design, we where sure that our network would be more powerful, than we need in the near future. Our applications / DB never used the full 1GBs in any way ... we loosing speed on the plain (painful LANCOM) switches and the applications (mostly Perl written in the beginning of the 2005).
But anyway, the network should be have enough capacity for the next years, because it is much more complicated to change network (design) components, than to kick a node.
With a 2nd NVMe in there you'd be at 2GB/s, or simple overkill. We would buy them ... so that in the end, every 12 disk has a separated NVMe
With decent SSDs and in-line journals (400GB DC S3610s) you'd be at 4.8 What about the worst case, two nodes are broken, fixed and replaced ? I red (a lot) that some Ceph users had massive problems, while the rebuild runs.
Of course if your I/O bandwidth needs are actually below 1GB/s at all times Very happy to put the finger in the wound, SPof ... is a very hard thing ... so we try to plan everything redundant :-)
The bad side of life: the SSD itself. A consumer SSD lays round about 70/80€, a DC SSD jumps up to 120-170€. My nightmare is: a lot of SSDs are jumping over the bridge at the same time .... -> arghh
But, we are working on it :-)
I've searching an alternative for the Asus board with more PCIe slots and maybe some components; better CPU with 3.5Ghz-> ; maybe a mix from the SSDs ...
At this time, I've found the X10DRi:
and I think we use the E5-2637v4 :-)
cu denny
|
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
