On 05/29/2015 04:47 PM, Samuel Just wrote:
Many people have reported that they need to lower the osd recovery config options to minimize the impact of recovery on client io. We are talking about changing the defaults as follows: osd_max_backfills to 1 (from 10) osd_recovery_max_active to 3 (from 15) osd_recovery_op_priority to 1 (from 10) osd_recovery_max_single_start to 1 (from 5) We'd like a bit of feedback first though. Is anyone happy with the current configs? Is anyone using something between these values and the current defaults? What kind of workload? I'd guess that lowering osd_max_backfills to 1 is probably a good idea, but I wonder whether lowering osd_recovery_max_active and osd_recovery_max_single_start will cause small objects to recover unacceptably slowly. Thoughts?
We ran recovery tests last year around when firefly was released. The basic gist of it was that as you increase client IO, the ratio of backfill to client IO changes for a given combination of priority settings. IE you can tune around 10 15 10 5, or 1 3 1 1, but in each case the ratio of client to recovery IO appears to scale with the amount of client IO, even past the super saturation point. I believe users will have a hard time finding optimal settings as clusters at the saturation point will behave differently than those in heavy super-saturation.
http://nhm.ceph.com/Ceph_3XRep_Backfill_Recovery_Results.pdf http://nhm.ceph.com/Ceph_62EC_Backfill_Recovery_Results.pdf _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
