Hi all, The osd recovery sleep option has been re-implemented to make it asynchronous. This value determines the sleep time in seconds before the next recovery or backfill op. We have done rigorous testing on HDDs, SSDs and HDD+SSD setups, with both Filestore and Bluestore, in order to come up with better default values for this configuration option. Detailed performance results can be found here: https://drive.google.com/file/d/0B7I5sSnjMhmbN1ZOanF3T2JIZm8/view?usp=sharing Following are some of our conclusions: - We need separate default values of osd_recovery_sleep for HDDs, SSDs and hybrid(HDD+SSD) setups. - In setups with only HDDs, increasing the amount of sleep time, shows performance improvement. However, the total time taken by background recovery operation also increases. We found that recovery sleep value of 0.1 sec is optimal for these kind of setups. - In setups with only SSDs, with increase in sleep value, we do not see any drastic improvement in performance. Therefore, we have decided to keep the sleep value 0, and not pay any extra price in terms of increased recovery time. - In hybrid setups, where osd data is on HDDs and osd journal is on SSDs, increasing the sleep value more than 0 helps, but we would like to choose a default value lesser than 0.1 sec in order to not increase the recovery time too much. We haven't finalized this value yet. Introducing this configuration option would require some more work, in terms of determining whether the journal is on HDD or SSD. With https://github.com/ceph/ceph/pull/16328, we are introducing two new configuration options osd_recovery_sleep_hdd and osd_recovery_sleep_ssd. Please let me know if you any thoughts about it or have trouble accessing the link. Thanks, Neha -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
