Re: Bluestore HDD Cluster Advice

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



If this is Skylake the 6 channel memory architecture lends itself better to configs such as 192GB (6 x 32) so yes even though 128GB is most likely sufficient usng (6 x 16GB) might be too small.

 

From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Martin Verges
Sent: Saturday, February 2, 2019 2:19 AM
To: John Petrini <jpetrini@xxxxxxxxxxxx>
Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>
Subject: Re: [ceph-users] Bluestore HDD Cluster Advice

 

Hello John,

 

you don't need such a big CPU, save yourself some money with a 12c/24t and invest it in better / more disks. Same goes for memory, 128G would be enough. Why do you install 4x 25G NIC, hard disks won't be able to use that capacity?

 

In addition, you can use the 2 disks for OSDs and not OS if you choose croit for system management meaning 10 more OSDs in your small cluster for better performance and a lot easier to manage. The best part of it, this feature comes with our complete free version, so it is just a gain on your side! Try it out

 

Please make sure to buy the right disks, there is a huge performance gap between 512e and 4Kn drives but near to no price difference. Bluestore does perform better then filestore in most environments, but as always depending on your specific workload. I would not recommend to even considering a filestore osd anymore, instead buy the correct hardware for your use case and configure the cluster accordingly.

 

--
Martin Verges
Managing director

Mobile: +49 174 9335695
E-Mail: martin.verges@xxxxxxxx
Chat: https://t.me/MartinVerges [t.me]

croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263

Web: https://croit.io [croit.io]
YouTube: https://goo.gl/PGE1Bx [goo.gl]

 

 

Am Fr., 1. Feb. 2019 um 18:26 Uhr schrieb John Petrini <jpetrini@xxxxxxxxxxxx>:

Hello,

We'll soon be building out four new luminous clusters with Bluestore.
Our current clusters are running filestore so we're not very familiar
with Bluestore yet and I'd like to have an idea of what to expect.

Here are the OSD hardware specs (5x per cluster):
2x 3.0GHz 18c/36t
22x 1.8TB 10K SAS (RAID1 OS + 20 OSD's)
5x 480GB Intel S4610 SSD's (WAL and DB)
192 GB RAM
4X Mellanox 25GB NIC
PERC H730p

With filestore we've found that we can achieve sub-millisecond write
latency by running very fast journals (currently Intel S4610's). My
main concern is that Bluestore doesn't use journals and instead writes
directly to the higher latency HDD; in theory resulting in slower acks
and higher write latency. How does Bluestore handle this? Can we
expect similar or better performance then our current filestore
clusters?

I've heard it repeated that Bluestore performs better than Filestore
but I've also heard some people claiming this is not always the case
with HDD's. Is there any truth to that and if so is there a
configuration we can use to achieve this same type of performance with
Bluestore?

Thanks all.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com [lists.ceph.com]

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux