Hi Ceph gurus,
I've got the following problem with our Ceph installation (Jewel): There
are various websites served from the CephFS mount. Sometimes, when I
copy many new (large?) files onto this mount, it seems that after a
certain delay, everything grinds to a halt. No websites are served;
processes are in D state; probably until Ceph has written everything to
disk. Then after a while, everythign recovers. Obviously, it would be
great if I could tune some values to make the experience more "even"
i.e. it can be a bit slower in general but OTOH without such huge
"spikes" in performance...
Now, first, I discovered there is "filestore flusher" documented here:
http://docs.ceph.com/docs/jewel/rados/configuration/filestore-config-ref/?highlight=flusher
Weirdly, when I use
ceph --admin-daemon /bla/bla config show
then I cannot see anything about this config option. Does it still exist?
Then I found this somewhat cryptic page:
http://docs.ceph.com/docs/jewel/dev/osd_internals/wbthrottle/
It says: "The flusher was not an adequate solution to this problem since
it forced writeback of small writes too eagerly killing performance."
Perhaps the "filestore flusher" was removed? But why is it still documented?
On the other hand, "config show" lists many "wbthrottle"-Options:
"filestore_wbthrottle_enable": "true",
"filestore_wbthrottle_xfs_bytes_hard_limit": "419430400",
"filestore_wbthrottle_xfs_bytes_start_flusher": "41943040",
"filestore_wbthrottle_xfs_inodes_hard_limit": "5000",
"filestore_wbthrottle_xfs_inodes_start_flusher": "500",
"filestore_wbthrottle_xfs_ios_hard_limit": "5000",
"filestore_wbthrottle_xfs_ios_start_flusher": "500",
I couldnt find them documented under docs.ceph.com, however they are
documented here:
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html/configuration_guide/file_store_configuration_reference
Quite confusing! Now, I wonder: Could/should I modify (raise/lower) some
of these values (we're using XFS)? Should I perhaps disable the
WBThrottle altogether for my use case?
Thank you,
Ranjan
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
