Hi Nana,
Great testing. One question, do we have any idea why # of
degraded object keep increasing during recovery? and especially in
fio-rbd testing, there are multiple spike on each test.
Xiaoxi
2017-07-20 7:56 GMT+08:00 Neha Ojha <nojha@xxxxxxxxxx>:
> Hi all,
>
> The osd recovery sleep option has been re-implemented to make it
> asynchronous. This value determines the sleep time in seconds before
> the next recovery or backfill op.
>
> We have done rigorous testing on HDDs, SSDs and HDD+SSD setups, with
> both Filestore and Bluestore, in order to come up with better default
> values for this configuration option. Detailed performance results can
> be found here: https://drive.google.com/file/d/0B7I5sSnjMhmbN1ZOanF3T2JIZm8/view?usp=sharing
>
> Following are some of our conclusions:
>
> - We need separate default values of osd_recovery_sleep for HDDs, SSDs
> and hybrid(HDD+SSD) setups.
>
> - In setups with only HDDs, increasing the amount of sleep time, shows
> performance improvement. However, the total time taken by background
> recovery operation also increases. We found that recovery sleep value
> of 0.1 sec is optimal for these kind of setups.
>
> - In setups with only SSDs, with increase in sleep value, we do not
> see any drastic improvement in performance. Therefore, we have decided
> to keep the sleep value 0, and not pay any extra price in terms of
> increased recovery time.
>
> - In hybrid setups, where osd data is on HDDs and osd journal is on
> SSDs, increasing the sleep value more than 0 helps, but we would like
> to choose a default value lesser than 0.1 sec in order to not increase
> the recovery time too much. We haven't finalized this value yet.
> Introducing this configuration option would require some more work, in
> terms of determining whether the journal is on HDD or SSD.
>
> With https://github.com/ceph/ceph/pull/16328, we are introducing two
> new configuration options osd_recovery_sleep_hdd and
> osd_recovery_sleep_ssd.
>
> Please let me know if you any thoughts about it or have trouble
> accessing the link.
>
> Thanks,
> Neha
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
