It's a beefy client with 32 cores. Less than 1% of the total CPU used on the client side. in the single rados case. And on the OSD node side, only 25% of the cpu being used, and disks and network are not at their limits -- Tom > -----Original Message----- > From: Mark Nelson [mailto:mnelson@xxxxxxxxxx] > Sent: Monday, May 11, 2015 9:23 AM > To: Deneau, Tom; ceph-devel > Subject: Re: rados bench single instance vs. multiple instances > > > > On 05/11/2015 09:18 AM, Deneau, Tom wrote: > > I have noticed the following while running rados bench seq read tests > > with a 40M object size > > > > single rados bench, 4 concurrent ops, bandwidth = > 190 MB/s > > 4 copies of rados bench, 1 concurrent op each, aggregate > > bandwidth = 310 MB/s > > > > and in fact the single rados bench seems limited to 190, no matter how many > concurrent ops. > > What's your CPU utilization like? > > > > > I don't see this kind of behavior with a 4M object size. > > > > (The above are with caches dropped on the osd targets) > > > > It doesn't seem to be related to the total number of bytes being > > processed by the single because if I don't drop the caches, both the > > single rados bench and the 4-copy rados bench get much higher numbers > > (600 vs. 900) but still the single rados bench appears limited, no matter > how many concurrent ops are used. > > > > Is there kind of throttling going on by design here? > > Nope, probably just client side inefficiency, especially if you are running > on slower cores. It's one of the reasons we tooled support for multiple > concurrent copies of rados bench into cbt. > > Mark > > > > > -- Tom Deneau, AMD > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" > > in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo > > info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
