Re: Quetions abount osd journal configuration

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



> On 26 Nov 2014, at 13:47, Christian Balzer <chibi@xxxxxxx> wrote:
> 
> On Wed, 26 Nov 2014 05:37:43 -0600 Mark Nelson wrote:
> 
>> On 11/26/2014 04:05 AM, Yujian Peng wrote:
> [snip]
>> 
>>> 
>>> Since the size of jornal partitions on SSDs is 10G, I want to set
>>> filestore max sync interval to 30 minutes. Is 30 minutes resonable?
>>> How to set filestore max sync interval on the fly?
>> 
>> The effect of this is that you'll be able to write the journal 
>> uninterrupted for a long time, but syncs could be very expensive.  I'd 
>> suggest giving it a try at a couple of values (default, 60s, 300s) under 
>> say 20 minutes of continuous heavy workload to observe the effects and 
>> see how you like it.
>> 
> 
> When I asked more or less the same question a while ago, I was left with
> the following information, please correct me if I'm wrong:
> 
> 1. "filestore max sync interval" is the longest time one can write to the
> journal, but flushes to the backing store can and will happen a lot
> earlier than that. What triggers these besides what I'm going to list
> below was left rather vague, though. If you want to enforce long periods
> of journal usage before writing out and hopefully coalescing things a lot,
> one needs to raise "filestore min sync interval". And the warnings about
> expensive syncs go double when doing that.

For any sync interval greater than ~30s you should also check and increase the kernel vm dirty buffer flushing period. See:

vm.dirty_background_ratio
vm.dirty_background_bytes
vm.dirty_ratio
vm.dirty_bytes
vm.dirty_writeback_centisecs
vm.dirty_expire_centisecs

Cheers, Dan

> 2. Once the journal is half full, a flush is triggered in any case. 
> 
> And once the journals are on SSDs, Yujian will likely discover that now
> his CPUs are the bottleneck in certain situations. ^o^
> 
> As for setting things on the fly, look for the "injectargs" syntax, like:
> ceph tell osd.* injectargs '--filestore_max_sync_interval 30'
> 
> 
> Christian
> -- 
> Christian Balzer        Network/Systems Engineer                
> chibi@xxxxxxx   	Global OnLine Japan/Fusion Communications
> http://www.gol.com/
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux