Hi Mark,
just taking the liberty to follow up on this one, as I'd really like to
get to the bottom of this.
On 28/11/2018 16:53, Florian Haas wrote:
> On 28/11/2018 15:52, Mark Nelson wrote:
>> Option("bluestore_default_buffered_read", Option::TYPE_BOOL,
>> Option::LEVEL_ADVANCED)
>> .set_default(true)
>> .set_flag(Option::FLAG_RUNTIME)
>> .set_description("Cache read results by default (unless hinted
>> NOCACHE or WONTNEED)"),
>>
>> Option("bluestore_default_buffered_write", Option::TYPE_BOOL,
>> Option::LEVEL_ADVANCED)
>> .set_default(false)
>> .set_flag(Option::FLAG_RUNTIME)
>> .set_description("Cache writes by default (unless hinted NOCACHE or
>> WONTNEED)"),
>>
>>
>> This is one area where bluestore is a lot more confusing for users that
>> filestore was. There was a lot of concern about enabling buffer cache
>> on writes by default because there's some associated overhead
>> (potentially both during writes and in the mempool thread when trimming
>> the cache). It might be worth enabling bluestore_default_buffered_write
>> and see if it helps reads.
>
> So yes this is rather counterintuitive, but I happily gave it a shot and
> the results are... more head-scratching than before. :)
>
> The output is here: http://paste.openstack.org/show/736324/
>
> In summary:
>
> 1. Write benchmark is in the same ballpark as before (good).
>
> 2. Read benchmark *without* readahead is *way* better than before
> (splendid!) but has a weird dip down to 9K IOPS that I find
> inexplicable. Any ideas on that?
>
> 3. Read benchmark *with* readahead is still abysmal, which I also find
> rather odd. What do you think about that one?
These two still confuse me.
And in addition, I'm curious as to what you think of the approach to
configure OSDs with bluestore_cache_kv_ratio = .49, so that rather
than using 1%/99%/0% of cache memory for metadata/KV data/objects, the
OSDs use 1%/49%/50%. Is this sensible? I assume the default of not using
any memory to actually cache object data is there for a reason, but I am
struggling to grasp what that reason would be. Particularly since in
filestore, we always got in-memory object caching for free, via the page
cache.
Thanks again!
Cheers,
Florian
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
