Re: Hardware configuration for OSD in a new all flash Ceph cluster

Warren Wang <Warren.Wang@xxxxxxxxxxx> · Thu, 2 Aug 2018 19:15:07 +0000

I would recommend a dedicated faster low latency device for journaling. If cost is an issue, you can try to swap the 2 CPUs for a single CPU, like the 5120. This also gets you out of any NUMA related issues. The reigning king for Ceph journals
 is dead (Intel P3700), but there are a few other options out there, including some NVMe options from Micron, like the 9200 Max.

Warren

From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> on behalf of Réal Waite <Real.Waite@xxxxxxxxxxxxx>

Date: Thursday, August 2, 2018 at 2:02 PM

To: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>

Subject: EXT: [ceph-users] Hardware configuration for OSD in a new all flash Ceph cluster

Hello,

We'd liked to setup a Ceph Cluster for IOPS-optimized Workloads. Our needs are for object storage (S3A for Spark, Boto for Python notebooks,…), RBD and, eventually, CephFS workloads.
Trough different readings for IOPS-optimized Ceph Workloads, we think to buy this kind of servers for the OSD:

Dell R740 with Chassis with up to 16 X 2.5” drive

2 x Intel® Xeon® Silver 4116 2.1G, 12C/24T, 9.6GT/s, 16M Cache, Turbo, HT (85W) DDR4-2400

8 x 16GB RDIMM, 2666MT/s, Dual Rank

HBA330 Controller, 12Gbps Adapter, Low Profile

16 x 1.92TB SSD SAS Mix Use 12Gbps 512n 2.5in Hot-plug Drive, PX05SV,3 DWPD,10512 TBW

OS disk = BOSS controller card + with 2 M.2 Sticks 120G (RAID 1),FH

Broadcom 5720 QP 1Gb Network Daughter Card (Configuration interface)

Mellanox ConnectX-3 Pro Dual Port 40 GbE QSFP+ PCIE Adapter Full Height (Cluster and Client Interface)
We will used the latest stable Luminous Ceph release support by RedHat. Therefore, we will used the XFS filesystem with the journal co-located with OSDs on the same SSD.

We will begin with 9 OSD servers and we will use a 3X or, maybe, a 2X replication factor since it is a all flash Ceph cluster.

What do you think of this configuration?

Réal Waite

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com