Re: Ceph with high disk densities?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 10/07/2013 12:29 PM, Gruher, Joseph R wrote:

In our small test deployments (160 HDs and OSDs across 20 machines)
our performance is quickly bounded by CPU and memory overhead. These
are 2U machines with 2x 6-core Nehalem; and running 8 OSDs consumed
25% of the total CPU time. This was a cuttlefish deployment.

You might be interested in trying a more recent release.  We've implemented
the SSE4 CRC32c instruction for CPUs that support it, which dramatically
reduces CPU overhead during large sequential writes.  On a 4U box with 24
spinning disks and 8 SSDs (4 bay unused) this brought CPU usage down from
something like 80% to around 40% during large sequential writes if I'm
remembering correctly.  The choice of the underlying filesystem will also
affect CPU overhead.  BTRFS tends to be a bit more CPU intensive than say
EXT4.


Mark, what type of CPUs are in the 4U box (number of sockets, architecture, core count, frequency)?


That machine has 2 E5-2630L CPUs. 6 cores each at 2GHz. Throughput was around 2.2GB/s from the client (so about 4.4GB/s going to all of the disks and SSDs when you count journal writes).

Mark
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux