With default memory settings, the general rule is 1GB ram/1TB OSD. If you have a 4TB OSD, you should plan to have at least 4GB ram. This was the recommendation for filestore OSDs, but it was a bit much memory for the OSDs. From what I've seen, this rule is a little more appropriate with bluestore now and should still be observed.
Please note that memory usage in a HEALTH_OK cluster is not the same amount of memory that the daemons will use during recovery. I have seen deployments with 4x memory usage during recovery.
On Thu, Mar 1, 2018 at 8:11 AM Stefan Kooman <stefan@xxxxxx> wrote:
Quoting Caspar Smit (casparsmit@xxxxxxxxxxx):
> Stefan,
>
> How many OSD's and how much RAM are in each server?
Currently 7 OSDs, 128 GB RAM. Max wil be 10 OSDs in these servers. 12
cores (at least one core per OSD).
> bluestore_cache_size=6G will not mean each OSD is using max 6GB RAM right?
Apparently. Sure they will use more RAM than just cache to function
correctly. I figured 3 GB per OSD would be enough ...
> Our bluestore hdd OSD's with bluestore_cache_size at 1G use ~4GB of total
> RAM. The cache is a part of the memory usage by bluestore OSD's.
A factor 4 is quite high, isn't it? Where is all this RAM used for
besides cache? RocksDB?
So how should I size the amount of RAM in a OSD server for 10 bluestore SSDs in a
replicated setup?
Thanks,
Stefan
--
| BIT BV http://www.bit.nl/ Kamer van Koophandel 09090351
| GPG: 0xD14839C6 +31 318 648 688 / info@xxxxxx
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
