BTW, you can completely turn off these throttles ( other than the filestore throttle ) by setting the value to 0. Thanks & Regards Somnath -----Original Message----- From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Deneau, Tom Sent: Tuesday, October 13, 2015 8:55 AM To: Sage Weil Cc: ceph-devel@xxxxxxxxxxxxxxx Subject: RE: throttles > -----Original Message----- > From: Sage Weil [mailto:sage@xxxxxxxxxxxx] > Sent: Tuesday, October 13, 2015 7:44 AM > To: Deneau, Tom > Cc: ceph-devel@xxxxxxxxxxxxxxx > Subject: Re: throttles > > On Mon, 12 Oct 2015, Deneau, Tom wrote: > > Looking at the perf counters on my osds, I see wait counts for the > > following throttle related perf counters: (This is from trying to > > benchmark using multiple rados bench client processes). > > > > throttle-filestore_bytes > > OPTION(filestore_queue_max_ops, OPT_INT, 50) > OPTION(filestore_queue_max_bytes, OPT_INT, 100 << 20) > > > throttle-msgr_dispatch_throttler-client > > OPTION(ms_dispatch_throttle_bytes, OPT_U64, 100 << 20) > > > throttle-osd_client_bytes > > throttle-osd_client_messages > > OPTION(osd_client_message_size_cap, OPT_U64, 500*1024L*1024L) // > client data allowed in-memory (in bytes) > OPTION(osd_client_message_cap, OPT_U64, 100) // num client > messages allowed in-memory > > > What are the config variables that would allow me to experiment with > these throttle limits? > > (When I look at the output from --admin-daemon osd.xx.asok config > > show, it's not clear which items these correspond to). > > These are all involved in slowing down clients to the rate of the > storage... > > sage Thanks, Sage and Somnath -- The reason I was interested is that I am seeing a levelling off of read bandwidth while the sar data shows we are not anywhere near the limits for cpu utilization (client or osd node) nor for disk utilization (no disk above 40% utilization) nor for network utilization (this is with 10Gig Ethernet). -- Tom -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
