> -----Original Message----- > From: Sage Weil [mailto:sage@xxxxxxxxxxxx] > Sent: Monday, May 11, 2015 12:04 PM > To: Deneau, Tom > Cc: ceph-devel > Subject: Re: rados bench single instance vs. multiple instances > > On Mon, 11 May 2015, Deneau, Tom wrote: > > I have noticed the following while running rados bench seq read tests > > with a 40M object size > > > > single rados bench, 4 concurrent ops, bandwidth = > 190 MB/s > > 4 copies of rados bench, 1 concurrent op each, aggregate > > bandwidth = 310 MB/s > > > > and in fact the single rados bench seems limited to 190, no matter how many > concurrent ops. > > > > I don't see this kind of behavior with a 4M object size. > > > > (The above are with caches dropped on the osd targets) > > > > It doesn't seem to be related to the total number of bytes being > > processed by the single because if I don't drop the caches, both the > > single rados bench and the 4-copy rados bench get much higher numbers > > (600 vs. 900) but still the single rados bench appears limited, no matter > how many concurrent ops are used. > > > > Is there kind of throttling going on by design here? > > It might be the librados throttles: > > OPTION(objecter_inflight_op_bytes, OPT_U64, 1024*1024*100) // max in-flight > data (both directions) > OPTION(objecter_inflight_ops, OPT_U64, 1024) // max in-flight > ios > > IIRC these only affect librados.. which would include 'rados bench'. > > sage > Just a follow-up that changing the limits of the two options mentioned above did indeed solve my problem. Also, my naïve understanding of the architecture was that things like RBD and RGW were layered on librados as shown in http://ceph.com/docs/master/architecture/. So wouldn't these throttles apply to those stacks as well? -- Tom Deneau -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
