In Addition you should Never use Consumer grade ssds for ceph as they will be reach the DWPD very soon...
Am 4. Januar 2018 09:54:55 MEZ schrieb "Rafał Wądołowski" <rwadolowski@xxxxxxxxxxxxxx>:
Hi folks,
I am currently benchmarking my cluster for an performance issue and I
have no idea, what is going on. I am using these devices in qemu.
Ceph version 12.2.2
Infrastructure:
3 x Ceph-mon
11 x Ceph-osd
Ceph-osd has 22x1TB Samsung SSD 850 EVO 1TB
96GB RAM
2x E5-2650 v4
4x10G Network (2 seperate bounds for cluster and public) with MTU 9000
I had tested it with rados bench:
# rados bench -p rbdbench 30 write -t 1
Total time run: 30.055677
Total writes made: 1199
Write size: 4194304
Object size: 4194304
Bandwidth (MB/sec): 159.571
Stddev Bandwidth: 6.83601
Max bandwidth (MB/sec): 168
Min bandwidth (MB/sec): 140
Average IOPS: 39
Stddev IOPS: 1
Max IOPS: 42
Min IOPS: 35
Average Latency(s): 0.0250656
Stddev Latency(s): 0.00321545
Max latency(s): 0.0471699
Min latency(s): 0.0206325
# ceph tell osd.0 bench
{
"bytes_written": 1073741824,
"blocksize": 4194304,
"bytes_per_sec": 414199397
}
Testing osd directly
# dd if=/dev/zero of=/dev/sdc bs=4M oflag=direct count=100
100+0 records in
100+0 records out
419430400 bytes (419 MB, 400 MiB) copied, 1.0066 s, 417 MB/s
When I do dd inside vm (bs=4M wih direct), I have result like in rados
bench.
I think that the speed should be arround ~400MB/s.
Is there any new parameters for rbd in luminous? Maybe I forgot about
some performance tricks? If more information needed feel free to ask.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
