[Rados Bench Result] Maximum bandwidth only 380MB/sec with 10Gb Network.

[Kuo Hugo <tonytkdk@xxxxxxxxx>]

Hi folks, 

I'm doing a test for Rados bench now. The cluster is deployed by ceph-deploy 

Ceph version: Emperor 

FS : XFS

I created a pool test3 with size1 : 

pool 13 'test3' rep size 1 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2000 pgp_num 2000 last_change 166 owner 0

The Rados bench command issued from an independent benchmark machine. 

Public network : 10GbE

Cluster network : 10GbE

Result : rados bench -p test3 30 write -t 100

Total time run:         30.275789

Total writes made:      2876

Write size:             4194304

Bandwidth (MB/sec):     379.974

Stddev Bandwidth:       90.2077

Max bandwidth (MB/sec): 420

Min bandwidth (MB/sec): 0

Average Latency:        1.03858

Stddev Latency:         0.112756

Max latency:            2.20545

Min latency:            0.10155

Why the bandwidth only 379MB/sec?   I expected the speed suppose to be over 900MB/sec due to the pool size only 1. 

There're 3 storage nodes here. 

10 OSDs on each storage node + journals on a SSD with 10 partitions.

Any suggestions? 

Thanks 

 

 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com