On 02/12/2018 19:48, Florian Haas wrote:
> Hi Mark,
>
> just taking the liberty to follow up on this one, as I'd really like to
> get to the bottom of this.
>
> On 28/11/2018 16:53, Florian Haas wrote:
>> On 28/11/2018 15:52, Mark Nelson wrote:
>>> Option("bluestore_default_buffered_read", Option::TYPE_BOOL,
>>> Option::LEVEL_ADVANCED)
>>> .set_default(true)
>>> .set_flag(Option::FLAG_RUNTIME)
>>> .set_description("Cache read results by default (unless hinted
>>> NOCACHE or WONTNEED)"),
>>>
>>> Option("bluestore_default_buffered_write", Option::TYPE_BOOL,
>>> Option::LEVEL_ADVANCED)
>>> .set_default(false)
>>> .set_flag(Option::FLAG_RUNTIME)
>>> .set_description("Cache writes by default (unless hinted NOCACHE or
>>> WONTNEED)"),
>>>
>>>
>>> This is one area where bluestore is a lot more confusing for users that
>>> filestore was. There was a lot of concern about enabling buffer cache
>>> on writes by default because there's some associated overhead
>>> (potentially both during writes and in the mempool thread when trimming
>>> the cache). It might be worth enabling bluestore_default_buffered_write
>>> and see if it helps reads.
>>
>> So yes this is rather counterintuitive, but I happily gave it a shot and
>> the results are... more head-scratching than before. :)
>>
>> The output is here: http://paste.openstack.org/show/736324/
>>
>> In summary:
>>
>> 1. Write benchmark is in the same ballpark as before (good).
>>
>> 2. Read benchmark *without* readahead is *way* better than before
>> (splendid!) but has a weird dip down to 9K IOPS that I find
>> inexplicable. Any ideas on that?
>>
>> 3. Read benchmark *with* readahead is still abysmal, which I also find
>> rather odd. What do you think about that one?
>
> These two still confuse me.
>
> And in addition, I'm curious as to what you think of the approach to
> configure OSDs with bluestore_cache_kv_ratio = .49, so that rather
> than using 1%/99%/0% of cache memory for metadata/KV data/objects, the
> OSDs use 1%/49%/50%. Is this sensible? I assume the default of not using
> any memory to actually cache object data is there for a reason, but I am
> struggling to grasp what that reason would be. Particularly since in
> filestore, we always got in-memory object caching for free, via the page
> cache.
Hi Mark,
do you mind if I give this another poke?
Cheers,
Florian
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
