On 04/05/2011 01:52 AM, Gregory Farnum wrote:
Data replicas don't service reads unless you specifically configure it for that (it doesn't provide POSIX semantics if you do that), and each replica needs to get the data written before it's considered safe. In your case you're doing a dd, which is a write to disk, so you're increasing how much data needs to get sent over the network and written to disk by increasing replication.
Without knowing more about your setup though we can't tell exactly why the performance is dropping as it is. Some possibilities:
1) You have a very small number of OSDs, so you're saturating the NIC on the primaries
2) You have a very small number of OSDs, so you're saturating the disk on all of them (this is particularly likely if you've got the journal on the same drive as the main store, since that's an automatic 50% cut in disk throughput).
3) Everything is connected via one cheap switch, which can't handle the traffic.
4) You've got some very slow OSDs, which will slow down each Placement Group they're a member of (and with higher replication levels you will hit them more often).
etc
Many thanks for your reply.
If I correct understood - the maximum write speed to the file system in
the ideal case is the network bandwidth from the client? Or bandwidth
divided by the number of copies?
How much OSDs recommended for 3 replicas?
P.S. without documentation have to ask stupid questions.:)
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
