On 04/22/2015 01:39 PM, Francois Lafont wrote:
Hi,
Christian Balzer wrote:
thanks for the feedback regarding the network questions. Currently I try
to solve the question of how much memory, cores and GHz for OSD nodes
and Monitors.
My research so far:
OSD nodes: 2 GB RAM, 2 GHz, 1 Core (?) per OSD
RAM is enough, but more helps (page cache on the storage node makes the
reads of hot objects quite fast and prevents concurrent access to the
disks).
Personally, I have seen a different rule for the RAM: "1GB for each 1TB
of OSD daemons". This is I understand in this doc:
http://ceph.com/docs/master/start/hardware-recommendations/#minimum-hardware-recommendations
So, for instance, with (it's just a stupid example):
- 4 OSD daemons of 6TB and
- 5 OSD daemons of 1TB
The needed RAM would be:
1GB x (4 x 6) + 1GB x (5 x 1) = 29GB for the RAM
Is it correct? Because if I follow the "2GB RAM per OSD" rule, I just need:
2GB x 9 = 18GB.
I'm not sure who came up with the 1GB for each 1TB of OSD daemons rule,
but frankly I don't think it scales well at the extremes. You can't get
by with 256MB of ram for OSDs backed by 256GB SSDs, nor do you need 6GB
of ram per OSD for 6TB spinning disks.
2-4GB of RAM per OSD is reasonable depending on how much page cache you
need. I wouldn't stray outside of that range myself.
Which rule is correct?
1GHz or so for per pure HDD based OSD, at least 2GHz for HDD OSDs with SSD
journals, as much as you can afford for entirely SSD based OSDs.
Are there links about the "at least 2Ghz per OSD with SSD journal", because
I have never seen that except in this mailing list. For instance in the
"HARDWARE CONFIGURATION GUIDE" of Inktank, it is just indicated: "one GHz
per OSD" (https://ceph.com/category/resources/).
Why should SSD journals increase the needed CPU?
What it really comes down to is that your CPU needs to be fast enough to
process your workload. Small IOs tend to be more CPU intensive than
large IOs. Some processors have higher IPC than others so it's all just
kind of a vague guessing game. With modern Intel XEON processors, 1GHz
of 1 core is a good general estimate. If you are doing lots of small IO
with SSD backed OSDs you may need more. If you are doing high
performance erasure coding you may need more. If you have slow disks
with journals on disk, 3x replication, and a mostly read workload, you
may be able to get away with less.
As always, the recommendations above are just recommendations. It's
best if you can test yourself.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
