In this case the spinners have their journals on an NVMe drive, 3 OSD : 1 NVMe Journal.
Will be trying tomorrow to get some benchmarks and compare some hdd/ssd/hybrid workloads to see performance differences across the three backing layers.
Most client traffic is read oriented to begin with, so keeping reads quick is likely the biggest goal here.
Appreciate everyone’s input and advice.
Reed
> On Apr 19, 2017, at 5:59 PM, Anthony D'Atri <aad@xxxxxxxxxxxxxx> wrote:
>
> Re ratio, I think you’re right.
>
> Write performance depends for sure on what the journal devices are. If the journals are colo’d on spinners, then for sure the affinity game isn’t going to help writes massively.
>
> My understanding of write latency is that min_size journals have to be written before the op returns, so if journals aren’t on SSD’s that’s going to be a big bottleneck.
>
>
>
>
>> Hi,
>>
>>>> Assuming production level, we would keep a pretty close 1:2 SSD:HDD ratio,
>>> 1:4-5 is common but depends on your needs and the devices in question, ie. assuming LFF drives and that you aren’t using crummy journals.
>>
>> You might be speaking about different ratios here. I think that Anthony is speaking about journal/OSD and Reed speaking about capacity ratio between and HDD and SSD tier/root.
>>
>> I have been experimenting with hybrid setups (1 copy on SSD + 2 copies on HDD), like Richard says you’ll get much better random read performance with primary OSD on SSD but write performance won’t be amazing since you still have 2 HDD copies to write before ACK.
>>
>> I know the doc suggests using primary affinity but since it’s a OSD level setting it does not play well with other storage tiers so I searched for other options. From what I have tested, a rule that selects the first/primary OSD from the ssd-root then the rest of the copies from the hdd-root works. Though I am not sure it is *guaranteed* that the first OSD selected will be primary.
>>
>> “rule hybrid {
>> ruleset 2
>> type replicated
>> min_size 1
>> max_size 10
>> step take ssd-root
>> step chooseleaf firstn 1 type host
>> step emit
>> step take hdd-root
>> step chooseleaf firstn -1 type host
>> step emit
>> }”
>>
>> Cheers,
>> Maxime
>>
>>
>>
>
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
