It doesn't sound like you're actually dealing with slow speeds when you execute the flush/evict commands, just that it isn't keeping your cache as small as you'd expect. What are your cache settings? This is all math and equations. One of your settings/variables must be off, skewing your result.
I use a cache pool in front of an EC pool, because that's still required in Jewel, but I don't need it for any speed performance of the underlying pool. I have my settings so that my pool remains empty pretty much constantly. I have a friend that has a similar setup, except his files are very likely to be read, modified, rewritten, etc for the next 24 hours once they are written, accessed, etc.
Here are my settings for my cache pool `ceph osd pool get {pool_name} all`. Notice my targets are 0 for full_ratio, max_objects, etc. Things stay in the cache because the cache_min_flush_age is 30 minutes and the cache_min_evict_age is an hour. My friend's settings vary only in that his min_flush_age and min_evict_age are 24 hours and 30 hours respectively. The default action as soon as an object reaches the minimum age is to be flushed and evicted because the target for the cache is to be completely empty. I wrote over 200GB of data to this EC pool yesterday and today there Is less than 500MB in its cache because of some minor reads that have been happening this morning. The cache pool is regularly 0% full without any intervention.
hit_set_type: bloom
hit_set_period: 3600
hit_set_count: 1
target_max_objects: 0
target_max_bytes: 2000000000000
cache_target_dirty_ratio: 0
cache_target_dirty_high_ratio: 0.6
cache_target_full_ratio: 0
cache_min_flush_age: 1800
cache_min_evict_age: 3600
min_read_recency_for_promote: 0
min_write_recency_for_promote: 1
On the other hand, if you are using cache tiering for actual performance tuning instead of just to satisfy the demands of an EC pool, then these settings probably don't make much sense for your use case.
On Wed, May 17, 2017 at 6:57 AM Ashley Merrick <ashley@xxxxxxxxxxxxxx> wrote:
Hello,
I recently doubles the PG's on my cache pool, from 256 to 512.
I have made sure all the PG's in question have been deep scrubbed however my cache no longer wish flush or evict at any decent speed.
If left to it's own it does a couple of objects here and there, but eventually grows and gets danjerouslt near its limit.
Currently I am having to run the cache flush / evict command in a loop which is very slowly bringing it down, burst of objects and then back to a few a second if lucky.
Both cache pool and underlying pool have plenty of capacity and performance is fine.
The only thing that has changed is amount of PG's and it's like somewhere in the flush calculation it hasn't updated with the new amount.
Have a max bytes limit and fairly Low flush and evict ratios to leave normally plenty of %
,Ashley
Sent from my iPhone
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
