H Greg,
Am 29.04.2014 22:23, schrieb Gregory Farnum:
On Tue, Apr 29, 2014 at 1:10 PM, Dan Van Der Ster
<daniel.vanderster@xxxxxxx> wrote:
Hi all,
Why is the default max sync interval only 5 seconds?
Today we realized what a huge difference that increasing this to 30 or 60s can do for the small write latency. Basically, with a 5s interval our 4k write latency is above 30-35ms and once we increase it to 30s we can get under 10ms (using spinning disks for journal and data.)
See the attached plot for the affect of this on a running cluster (the plot shows the max, avg, min write latency from a short rados bench every 10 mins). The change from 5s to 60s was applied at noon today. (And our journals are large enough, don't worry).
In the interest of having sensible defaults, is there any reason not to increase this to 30s?
If you've got reasonable confidence in the quality of your
measurements across the workloads you serve, you should bump it up.
Part of what might be happening here is simply that fewer of your
small-io writes are running into a sync interval.
I suspect that most users will see improvement by bumping up the
limits and occasionally agitate to change the defaults, but Sam has
always pushed back against doing so for reasons I don't entirely
recall. :) (The potential for a burstier throughput profile?)
-Greg
What is about those?
filestore queue max ops = 500
filestore_queue_committing_max_ops = 5000
filestore_queue_max_bytes = 419430400
filestore_queue_committing_max_bytes = 419430400
filestore_wbthrottle_xfs_bytes_start_flusher = 125829120
filestore_wbthrottle_xfs_bytes_hard_limit = 419430400
filestore_wbthrottle_xfs_ios_start_flusher = 5000
filestore_wbthrottle_xfs_ios_hard_limit = 50000
filestore_wbthrottle_xfs_inodes_start_flusher = 1000
filestore_wbthrottle_xfs_inodes_hard_limit = 10000
They should be adjusted too? right?
Stefan
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
