On 10/07/2013 12:29 PM, Gruher, Joseph R wrote:
In our small test deployments (160 HDs and OSDs across 20 machines)
our performance is quickly bounded by CPU and memory overhead. These
are 2U machines with 2x 6-core Nehalem; and running 8 OSDs consumed
25% of the total CPU time. This was a cuttlefish deployment.
You might be interested in trying a more recent release. We've implemented
the SSE4 CRC32c instruction for CPUs that support it, which dramatically
reduces CPU overhead during large sequential writes. On a 4U box with 24
spinning disks and 8 SSDs (4 bay unused) this brought CPU usage down from
something like 80% to around 40% during large sequential writes if I'm
remembering correctly. The choice of the underlying filesystem will also
affect CPU overhead. BTRFS tends to be a bit more CPU intensive than say
EXT4.
Mark, what type of CPUs are in the 4U box (number of sockets, architecture, core count, frequency)?
That machine has 2 E5-2630L CPUs. 6 cores each at 2GHz. Throughput was
around 2.2GB/s from the client (so about 4.4GB/s going to all of the
disks and SSDs when you count journal writes).
Mark
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
