Thank you both for your replies and for clearing up the matter. I totally understand that you can't actually know the size of a pool, I was just using the terminology to highlight the point in the 1st article that seems to suggest the relative option already knows this. But as you have confirmed the 2nd article is the correct behaviour. Please let me know if there is anything I can do to help regarding getting the docs updated. As a related side note, the cache tier changes coming in Hammer look great, keep up the good work guys. Thanks again, Nick -----Original Message----- From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Sage Weil Sent: 07 February 2015 20:57 To: Nick Fisk Cc: ceph-users@xxxxxxxxxxxxxx Subject: Re: Cache Settings On Sat, 7 Feb 2015, Nick Fisk wrote: > Hi All, > > Time for a little Saturday evening Ceph related quiz. > > >From this documentation page > > http://ceph.com/docs/master/rados/operations/cache-tiering/ > > It seems to indicate that you can either flush/evict using relative > sizing > (cache_target_dirty_ratio) or absolute sizing (target_max_bytes). But > the two are separate methods and are mutually exclusive. Ie flush at > 80% or when you hit the number of bytes specified No, target_max_bytes defines the 'target' in cache_target_dirty_ratio. Hmm, skimming that doc there are a few inaccuracies :(, putting it on my todo. > The same goes for the max_age parameters, ie it will flush all objects > older than 300 seconds no matter how full the pool is. Yes, but only when the agent happens across one. See below.. > However this documentation page > > https://ceph.com/docs/master/dev/cache-pool/ > > Seems to indicate that the target_max_bytes is actually the number of > bytes that the cache_target_dirty_ratio uses to calculate the size of > the cache pool it has to work with. And that the max_age parameters > just make sure objects aren't evicted too quickly. Yes > 1. Maybe, I'm reading it wrong but they appear conflicting to me, > Which is correct? #2 > The following questions may be invalid depending on the answer to #1 > > 2. Assuming link #1 is correct, is it possible to have multiple cache > pools on group of SSD's and how does Ceph work out capacity for each pool? Nope. See below.. > 3. Assuming #2 is correct, I can also specify min_age variables > without overriding the target_max_bytes and cache_target_dirty_ratio? Unless there is some corner case I'm missing, the agent will never activate unless you specify some size for the cache, either target_max_bytes or target_max_objects. Without the agent has no frame of reference. > So assuming link #2 is correct (which makes more sense to me), if I > had the following configuration > > target_max_bytes = 10,000,000,000 > cache_target_dirty_ratio = .4 > cache_full_dirty_ratio = .8 > cache_min_flush_age = 60 > cache_min_evict_age = 120 > > Then are the following assumptions true:- > > 1. I have a cache pool that is 10G total in size, regardless of the > actual size of the pool Yes. Note that "the actual size of the pool" is a very nebulous thing. Pools are composed of PGs that map to OSDs that have some amount of free space, but other pools' PGs may too. The closest thing to "size of the pool" that you get is the 'ceph df detail' 'max avail' column, which is an *estimation* of how much data you could write to teh pool. In the future Ceph could possibly use this (or something like it) to figure this out, but today it does not. > 2. When the pool has 4G of dirty bytes in it, it will start trying to > flush them as long as they are older than 60 seconds 3. When the pool > is 8G full it will start evicting all objects that are older than 120 > seconds in a LRU order At a high level, yes. But we don't maintain an LRU of objects (this would be too expensive to update for read-heavy workloads). Instead we use bloom filters to track which objects are been read, and when the agent activates (usage >= target_full_ratio) we iterate over objects in the pool and evict older objects. The min_age and max_age either force us to skip flush/evict or always evict *when we examine each object*. But they don't ensure that we will examine it right when they reach that age (esp since the agent is idle when the pool is mostly empty). > 4. If I manage to fill the pool up to 10G, Ceph will block until free > space becomes available from evictions Yes > 5. If I had a 100G worth of SSD capacity after replication, I could > have 10 of these cache pools (disregard performance concerns) Yep! Hope that helps! sage _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |