On Tue, Feb 21, 2012 at 1:50 PM, Noah Watkins <jayhawk@xxxxxxxxxxx> wrote: > > On Feb 21, 2012, at 12:40 PM, Gregory Farnum wrote: > >> >> That's pretty much expected. -t is not actually controlling the number >> of threads, but the number of requests the rados tool will dispatch at >> a time. With -t 1 you're telling it to send a single request, wait for >> it to commit, then dispatch another request. (ie, latency bound) With >> -t 10 you are dispatching 10 requests and dispatching a new one >> whenever one of those comes back. (ie, mostly bandwidth bound) > > Should the expectation be 30 MB/s (the slowest disk) or 1/3 of that for triplication? I was under the impression that the ack came back for being in memory on 2 of the 3 nodes. Acks come back once they're in memory on all nodes. Commits come back once they're on disk in all nodes. Doing 2 out of 3 is not fail-safe if you lose the two nodes the data is on. *shrug* I believe the rados bench requires commits, not acks, but I could be misremembering. In any case, working out the expected speeds on a cluster with heterogenous disks is a challenge we haven't fully automated yet. If you've got 3 nodes with 3x replication and one node can only handle 30MB/s though, that's what you're going to get in terms of asymptotic throughput, yes. In a 23-node cluster it's considerably more complicated and I'd have to draw it out. -Greg (not sure if that answered your question or not...) -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
