Hello, On Wed, 6 Sep 2017 09:09:54 -0400 Alex Gorbachev wrote: > We are planning a Jewel filestore based cluster for a performance > sensitive healthcare client, and the conservative OSD choice is > Samsung SM863A. > While I totally see where you're coming from and me having stated that I'll give Luminous and Bluestore some time to mature, I'd also be looking into that if I were being in the planning phase now, with like 3 months before deployment. The inherent performance increase with Bluestore (and having something that hopefully won't need touching/upgrading for a while) shouldn't be ignored. The SSDs are fine, I've been starting to use those recently (though not with Ceph yet) as Intel DC S36xx or 37xx are impossible to get. They're a bit slower in the write IOPS department, but good enough for me. I'm sure you considered this, but you really want to have a good grip on the data (write) volume, since with in-line journals and other overheads those 3.6 DWPD endurance is going to be something closer to 1, 1.5 at best. > I am going to put an 8GB Areca HBA in front of it to cache small > metadata operations, I see we share the same taste in HW, I vastly prefer Areca over anything Adaptec (whatever their latest owner is) or LSI (same) have ever done. An 8GB cache will be helpful, but it boils down to how much data and IOPS are going through it and how many SSDs are behind it. At least with JBOD SSDs as the storage, the performance hit won't be as dramatic compared to a HDD based RAID6 when the cache is overloaded. > but was wondering if anyone has seen a positive > impact from also using PCIe journals (e.g. Intel P3700 or even the > older 910 series) in front of such SSDs? > NVMe journals (or WAL and DB space for Bluestore) are nice and can certainly help, especially if Ceph is tuned accordingly. Avoid non DC NVMes, I doubt you can still get 910s, they are officially EOL. You want to match capabilities and endurances, a DC P3700 800GB would be an OK match for 3-4 SM863a 960GB for example. There are people here who have actually done this, hopefully some will speak up. Christian -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Rakuten Communications _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |