We are looking at 100+ nodes.
I know that the Ceph official recommendation is 1GB of RAM per 1TB of disk. Was this ever changed since 2015?
CERN is definitely using less (source: https://cds.cern.ch/record/2015206/files/CephScaleTestMarch2015.pdf)
RedHat suggests using 16GB + 2GB/HDD as the latest requirements.
BTW: Anyone has comments on SSD sizes for Bluestore or the other questions?
Jan
2018-03-09 14:58 GMT+01:00 Brady Deetz <bdeetz@xxxxxxxxx>:
I'd increase ram. 1GB per 1TB of disk is the recommendation.Another thing you need to consider is your node density. 12x10TB is a lot of data to have to rebalance if you aren't going to have 20+ nodes. I have 17 nodes with 24x6TB disks each. Rebuilds can take what seems like an eternity. It may be worth looking at cheaper sockets and smaller disks in order to increase your node count.How many nodes will this cluster have?On Mar 9, 2018 4:16 AM, "Ján Senko" <jan.senko@xxxxxxxxx> wrote:______________________________I am planning a new Ceph deployement and I have few questions that I could not find good answers yet.Our nodes will be using Xeon-D machines with 12 HDDs each and 64GB each.Our target is to use 10TB drives for 120TB capacity per node.1. We want to have small amount of SSDs in the machines. For OS and I guess for WAL/DB of Bluestore. I am thinking about having a RAID 1 with two 400GB 2.5" SSD drives. Will this fit WAL/DB? We plan to store many small objects.2. While doing scrub/deep scrub, is there any significant network traffic? Assuming we are using Erasure coding pool, how do the nodes check the consistency of an object? Do they transfer the whole object chunks or do they only transfer the checksums?3. We have to decide on which HDD to use, and there is a question of HGST vs Seagate, 512e vs 4kn sectors, SATA vs SAS. Do you have some tips for these decisions? We do not have very high IO, so we do not need performance at any cost. As for manufacturer and the sector size, I haven't found any guidelines/benchmarks that would steer me towards any.Thank you for your insightJan
_________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Jan Senko, Skype janos-
Phone in Switzerland: +41 774 144 602
Phone in Czech Republic: +420 777 843 818
Phone in Switzerland: +41 774 144 602
Phone in Czech Republic: +420 777 843 818
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
