On Tue, Feb 21, 2012 at 12:25 PM, Noah Watkins <jayhawk@xxxxxxxxxxx> wrote: > Hi, we are trying to track down a performance problem with rados that is leading no where. > > We are using 23 OSDs. Running "ceph osd tell <osd> bench" we are seeing decent performance on each osd. > > However, running "rados -p rbd bench 120 write -t 1" from a client we only see around 10 MBs. We can increase the number of threads from 1 to 10 and see an increase in 10 MB/s for each new thread up until the network bottlenecks us around 100 MB/s. > > None of the micro benchmarks showed throughput less than 30 MB/s, so we are a bit at a loss for tracking down this performance issue. > > Any suggestions? That's pretty much expected. -t is not actually controlling the number of threads, but the number of requests the rados tool will dispatch at a time. With -t 1 you're telling it to send a single request, wait for it to commit, then dispatch another request. (ie, latency bound) With -t 10 you are dispatching 10 requests and dispatching a new one whenever one of those comes back. (ie, mostly bandwidth bound) -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
