On Mon, May 16, 2016 at 12:20 PM, Nick Fisk <nick@xxxxxxxxxx> wrote:
> -----Original Message-----
> From: Peter Kerdisle [mailto:peter.kerdisle@xxxxxxxxx]
You might want to rethink that slightly. Keeping in mind that with EC pools currently any write will force a promotion and then dirty the object. If you are then almost immediately flushing the objects back down, you are going to end up with a lot of amplification for writes. You want to keep dirty objects in the cache pool so that you don't incur this penalty if they are going to be written to again in the near future.> Sent: 16 May 2016 11:04
> To: Nick Fisk <nick@xxxxxxxxxx>
> Cc: ceph-users@xxxxxxxxxxxxxx
> Subject: Re: Erasure pool performance expectations
>
>
> On Mon, May 16, 2016 at 11:58 AM, Nick Fisk <nick@xxxxxxxxxx> wrote:
> > -----Original Message-----
> > From: Peter Kerdisle [mailto:peter.kerdisle@xxxxxxxxx]
> > Sent: 16 May 2016 10:39
> > To: nick@xxxxxxxxxx
> > Cc: ceph-users@xxxxxxxxxxxxxx
> > Subject: Re: Erasure pool performance expectations
> >
> > I'm forcing a flush by lower the cache_target_dirty_ratio to a lower value.
> > This forces writes to the EC pool, these are the operations I'm trying to
> > throttle a bit. I am understanding you correctly that's throttling only works
> for
> > the other way around? Promoting cold objects into the hot cache?
>
> Yes that’s correct. You want to throttle the flushes which is done by another
> setting(s)
>
> Firstly set something like this in your ceph.conf
> osd_agent_max_low_ops = 1
> osd_agent_max_ops = 4
> I did not know about this, that's great, I will play around with these.
>
>
> This controls how many parallel threads the tiering agent will use. You can
> bump them up later if needed.
>
> Next set on your cache pools, these two settings. Try and keep them about
> .2 apart. So something like .4 and .6 are good to start with.
> cache_target_dirty_ratio
> cache_target_dirty_high_ratio
> Here is actually the heart of the matter. Ideally I would love to run it at 0.0 if
> that makes sense. I want no dirty objects in my hot cache at all, has anybody
> ever tried this? Right now I'm just pushing cache_target_dirty_ratio during
> low activity moments by setting it to 0.2 and then bringing it back up to 0.6
> when it's done or activity starts up again.
After reading this again it does raise an other question. I'm not sure I understood promotions correctly before this response. A promotion is the act of moving a piece from the EC pool onto the cache tier, is this correct? If so does a promotion remove the actual data from the EC pool and make it dirty again? I was under the impression it was still on the EC pool but also on the cache tier at that point.
I'm guessing what you want is a buffer, so that you can have bursts of activity without incurring the performance penalty of flushing? That’s hopefully what the high and low flushes should give you. By setting to .4 and .6, you will have a .2x"Cache Tier Capacity" buffer of cache tier space, where only slow flushing will occur.
This is indeed one of the reasons. The other reason was that I thought that by removing dirty objects I didn't need replication on the cache tier, which I'm now starting to doubt again...
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
