Deep scrub debug option

[Broccoli Bob <brockolibob@xxxxxxxxx>]

I have been running a ceph cluster for a while and one of the main things that impacts performance is deep-scrubbing. I would like to limit this as much as possible and have tried the below options to do this:

osd scrub sleep = 1                      # Time to sleep before scrubbing next group of chunks
osd scrub chunk max = 1             # Maximum number of chunks to scrub during a single operation
osd scrub chunk min = 1              # Minimum number of chunks to scrub during a single operation
osd scrub max pre-emption = 30 # Maximum number of times ceph pre-empts a deep scrub due to a client operation blocking it
osd client op priority =  63            # The priority of client operations
osd requested scrub priority = 1       # The priority of administrator requested scrubs
osd scrub priority = 1                    # The priority of scheduled scrubs

These options combined do slow down scrubbing and make it easier on client IO, but there is still a performance impact when they happen. One other option I have found is:

‘osd debug deep scrub sleep’

 

Setting this option to even 0.1 will have an immediate and large effect (I have seen nodes that were scrubbing at 50MB/s to down to <5MB/s). The effect is larger than all the above options combined and is exactly what I want, however there is no documentation about this option and I don’t know the full impact of configuring it. It also spams the logs with messages saying that the deep scrub is sleeping for x seconds. So I would like to know:

1. Is it safe to have the debug deep scrub option turned on? If so is there a way to stop it from spamming the logs?
2. If it is not safe are there any other options available to us to limit the impact of deep scrubbing?

 Thank you and Regards

 

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx