Hi, > On 26 Nov 2014, at 17:07, Yujian Peng <pengyujian5201314@xxxxxxx> wrote: > > > Thanks a lot! > IOPS is a bottleneck in my cluster and the object disks are much slower than > SSDs. I don't know whether SSDs will be used as caches if > filestore_max_sync_interval is set to a big value. I will set > filestore_max_sync_interval to a couple of value to observe the effect. > > If filestore_max_sync_interval is greater than 30s, how to set kernel vm dirty > buffer? > In the past I was doing some test to try and completely eliminate all background flushing to the OSD devices. For this, I did something like: filestore max sync interval = 120 filestore min sync interval = 119 vm.dirty_background_ratio = 40 vm.dirty_background_bytes = 0 vm.dirty_ratio = 40 vm.dirty_bytes = 0 vm.dirty_writeback_centisecs = 500 vm.dirty_expire_centisecs = 3000 Given those settings, you should then run a test and check iostat -xm 1. You should see writes on the journals, but no writes on the OSD devices. (If you increase the debug_filestore level to 10 or 20 you can also see exactly when the filestore is sync’d, and correlate that with what you see in iostat). Following this test you can get an idea how the iops perform when the SSDs alone are used for writes. For production, with 120s filestore sync interval, you can probably live with something like: filestore max sync interval = 120 filestore min sync interval = <the default> vm.dirty_background_ratio = 10 vm.dirty_background_bytes = 0 vm.dirty_ratio = 40 vm.dirty_bytes = 0 vm.dirty_writeback_centisecs = 500 vm.dirty_expire_centisecs = 1200 Please refer to the doc to get a full understanding of how to tune those values: https://www.kernel.org/doc/Documentation/sysctl/vm.txt Cheers, Dan _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |