Hello,
we are brand new to Ceph and planing it as our future storage for
KVM/LXC VMs as replacement for Xen / DRBD / Pacemaker / Synology (NFS)
stuff.
We have two goals:
* High availability
* Short latency for our transaction services
* For later: replication to different datacenter connected via 10Gb/s FC
Our services are:
* Webapplication as frontent
* Database (Sybase / MariaDB Galera) as backend
All needed for doing transactions
All we are planing is at this time more than we need, but for future
development and replacement for our old hardware stuff and software, we
want the best, we can get for our (approved) money :-)
So, here we are:
Starting with a six OSD node cluster, that are doing not only OSD stuff,
but also holding the mon services. We want to store data only via API so
a separated meta server isn't needed, as I understand all the documents
right.
The first test hardware is:
*Motherboard: Asus Z10Pr-D16
**
https://www.asus.com/de/Commercial-Servers-Workstations/Z10PRD16/specifications/
* CPU: 2 x E5-2620v4
* Ram: 4 x 32GB DDR4 2400MHz
* Chassis: RSC-2AT0-80PG-SA3C-0BL-A
** http://www.aicipc.com/ProductSKU.aspx?ref=RSC-2AT
** Edition without Expander
* SAS: 1 x 9305-24i
**
http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-9305-24i#specifications
* Storage NIC: 1 x Infiniband MCX314A-BCCT
** I red, that ConnectX-3 Pro is better supported, than the X-4 and a
bit cheaper
** Switch: 2 x Mellanox SX6012 (56Gb/s)
** Active FC cables
** Maybe VPI is nice to have, but unsure.
* Production NIC: 1 x Intel 520 dual port SFP+
** Connected each to one of a HP 2920 10Gb/s ports via 802.3ad
All nodes are connected over cross to every switch, so if one switch
goes down, a second path is available.
* Disk:
** Storage: 24 x Crucial MX300 250GB (maybe for production 12xSSD / 12x
big Sata disks)
** OSD journal: 1 x Intel SSD DC P3700 PCIe
One of the hardest part was the chassis with or without active expander,
so that we can use a "cheaper" HBA, like the 8i or something else.
Also if we want/need a full raid controller like the Megaraid
sas-9361-8i, because of battery and cache. But it seems, that it isn't
really needed in our case. Sure, the cache is one of the benefits, but
maybe it is more complicated, than a plain HBA.
From the Ceph point of view, we want, that two OSD nodes can go down in
a worst case scenario, but keeping our business up (a bit slower is OK,
and expected). Also if the nodes comes back, we are not down, because of
the replication stuff ;-)
The OS would be Proxmox 4.x (based on Debian Jessie) with Hammer or
Jewel, but WITHOUT ANY VMs on it. We want to keep the systems are in one
hand :-)
So we want to know, the hardware should be O.K also with running the mon
servers on the same HW, like the OSDs. We know, that every OSD should
own a core, so the 2620v4 has 8 cores, with HT 16 and in sum we have 32
CPUs per OSD node, which should be fine, .... I think ....
It would be very helpful, if someone take a short view on our list, if
there is component we shouldn't buy for the production side of life :-)
cu denny
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
