Re: New cluster - configuration tips and reccomendation - NVMe

["Van Leeuwen, Robert" <rovanleeuwen@xxxxxxxx>]

Hi Max,

 

You might also want to look at the PCIE lanes.

I am not an expert on the matter but my guess would be the 8 NVME drives + 2x100Gbit would be too much for

the current Xeon generation (40 PCIE lanes) to fully utilize.

 

I think the upcoming AMD/Intel offerings will improve that quite a bit so you may want to wait for that.
As mentioned earlier. Single Core cpu speed matters for latency so you probably want to up that.

You can also look at the DIMM configuration.

TBH I am not sure how much it impacts Ceph performance but having just 2 DIMMS slots populated will not give you max memory bandwidth.

Having some extra memory for read-cache probably won’t hurt either (unless you know your workload won’t include any cacheable reads)

 

Cheers,

Robert van Leeuwen

 

From: ceph-users <ceph-users-bounces@xxxxxxxxxxxxxx> on behalf of Massimiliano Cuttini <max@xxxxxxxxxxxxx>
Organization: PhoenixWeb Srl
Date: Wednesday, July 5, 2017 at 10:54 AM
To: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
Subject: [ceph-users] New cluster - configuration tips and reccomendation - NVMe

 

Dear all,

luminous is coming and sooner we should be allowed to avoid double writing.
This means use 100% of the speed of SSD and NVMe.
Cluster made all of SSD and NVMe will not be penalized and start to make sense.

Looking forward I'm building the next pool of storage which we'll setup on next term.
We are taking in consideration a pool of 4 with the following single node configuration:

• 2x E5-2603 v4 - 6 cores - 1.70GHz
• 2x 32Gb of RAM
• 2x NVMe M2 for OS
• 6x NVMe U2 for OSD
• 2x 100Gib ethernet cards

We have yet not sure about which Intel and how much RAM we should put on it to avoid CPU bottleneck.
Can you help me to choose the right couple of CPU?
Did you see any issue on the configuration proposed?

Thanks,
Max

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com