performance in a small cluster

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

we have a small cluster at a customer's site with three nodes and 4 SSD-OSDs each.
Connected with 10G the system is supposed to perform well.

rados bench shows ~450MB/s write and ~950MB/s read speeds with 4MB objects but only 20MB/s write and 95MB/s read with 4KB objects.

This is a little bit disappointing as the 4K performance is also seen in KVM VMs using RBD.

Is there anything we can do to improve performance with small objects / block sizes?

Jumbo frames have already been enabled.

4MB objects write:

Total time run:         30.218930
Total writes made:      3391
Write size:             4194304
Object size:            4194304
Bandwidth (MB/sec):     448.858
Stddev Bandwidth:       63.5044
Max bandwidth (MB/sec): 552
Min bandwidth (MB/sec): 320
Average IOPS:           112
Stddev IOPS:            15
Max IOPS:               138
Min IOPS:               80
Average Latency(s):     0.142475
Stddev Latency(s):      0.0990132
Max latency(s):         0.814715
Min latency(s):         0.0308732

4MB objects rand read:

Total time run:       30.169312
Total reads made:     7223
Read size:            4194304
Object size:          4194304
Bandwidth (MB/sec):   957.662
Average IOPS:         239
Stddev IOPS:          23
Max IOPS:             272
Min IOPS:             175
Average Latency(s):   0.0653696
Max latency(s):       0.517275
Min latency(s):       0.00201978

4K objects write:

Total time run:         30.002628
Total writes made:      165404
Write size:             4096
Object size:            4096
Bandwidth (MB/sec):     21.5351
Stddev Bandwidth:       2.0575
Max bandwidth (MB/sec): 22.4727
Min bandwidth (MB/sec): 11.0508
Average IOPS:           5512
Stddev IOPS:            526
Max IOPS:               5753
Min IOPS:               2829
Average Latency(s):     0.00290095
Stddev Latency(s):      0.0015036
Max latency(s):         0.0778454
Min latency(s):         0.00174262

4K objects read:

Total time run:       30.000538
Total reads made:     1064610
Read size:            4096
Object size:          4096
Bandwidth (MB/sec):   138.619
Average IOPS:         35486
Stddev IOPS:          3776
Max IOPS:             42208
Min IOPS:             26264
Average Latency(s):   0.000443905
Max latency(s):       0.0123462
Min latency(s):       0.000123081


Regards
--
Robert Sander
Heinlein Support GmbH
Linux: Akademie - Support - Hosting
http://www.heinlein-support.de

Tel: 030-405051-43
Fax: 030-405051-19

Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein  -- Sitz: Berlin
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux