Documented here: https://github.com/ceph/ceph/blob/9754cafc029e1da83f5ddd4332b69066fe6b3ffb/src/common/options/global.yaml.in#L3202 Introduced back here with a bunch of other scrub tweaks: https://github.com/ceph/ceph/pull/18971/files Are your OSDs HDDs? Using EC? How many deep scrubs do you have running in parallel? Assuming more than one, you could increase osd_deep_scrub_interval to spread them out over time. > On Feb 7, 2023, at 05:58, Broccoli Bob <brockolibob@xxxxxxxxx> wrote: > > I have been running a ceph cluster for a while and one of the main things that impacts performance is deep-scrubbing. I would like to limit this as much as possible and have tried the below options to do this: > > osd scrub sleep = 1 # Time to sleep before scrubbing next group of chunks > osd scrub chunk max = 1 # Maximum number of chunks to scrub during a single operation > osd scrub chunk min = 1 # Minimum number of chunks to scrub during a single operation > osd scrub max pre-emption = 30 # Maximum number of times ceph pre-empts a deep scrub due to a client operation blocking it > osd client op priority = 63 # The priority of client operations > osd requested scrub priority = 1 # The priority of administrator requested scrubs > osd scrub priority = 1 # The priority of scheduled scrubs > > These options combined do slow down scrubbing and make it easier on client IO, but there is still a performance impact when they happen. One other option I have found is: > ‘osd debug deep scrub sleep’ > > Setting this option to even 0.1 will have an immediate and large effect (I have seen nodes that were scrubbing at 50MB/s to down to <5MB/s). The effect is larger than all the above options combined and is exactly what I want, however there is no documentation about this option and I don’t know the full impact of configuring it. It also spams the logs with messages saying that the deep scrub is sleeping for x seconds. So I would like to know: > Is it safe to have the debug deep scrub option turned on? If so is there a way to stop it from spamming the logs? > If it is not safe are there any other options available to us to limit the impact of deep scrubbing? > Thank you and Regards > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx <mailto:ceph-users@xxxxxxx> > To unsubscribe send an email to ceph-users-leave@xxxxxxx <mailto:ceph-users-leave@xxxxxxx> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |