Hi Roozbeh,
There isn't nearly enough information here regarding your benchmark and
test parameters to be able to tell why you are seeing performance
swings. It could be anything from network hiccups, to throttling in the
ceph stack, to unlucky randomness in object distribution, to vibrations
in the rack causing your disk heads to resync, to fragmentation of the
underlying filesystem (especially important for sequential reads).
Generally speaking if you want to try to isolate the source of the
problem, it's best to find a way to make the issue repeatable on demand,
then setup your tests so you can record system metrics (device
queue/service times, throughput stalls, network oddities, etc) and start
systematically tracking down when and why slowdowns occur. Sometimes
you might even be able to reproduce issues outside of Ceph (Network
problems are often a common source).
It might also be worth looking at your PG and data distribution. IE if
you have some clumpiness you might see variation in performance as some
OSDs starve for IOs while others are overloaded.
Good luck!
Mark
On 05/03/2016 11:16 AM, Roozbeh Shafiee wrote:
Hi,
I have a test Ceph cluster in my lab which will be a storage backend for
one of my projects.
This cluster is my first experience on CentOS-7, but recently I had some
use case on Ubuntu 14.04 too.
Actually everything works fine and I have a good functionality on this
cluster, but the main problem is the performance
of cluster in read and write data. I have too much swing in read and
write and the rate of this swing is between 60 KB/s - 70 MB/s, specially
on read.
how can I tune this cluster as stable storage backend for my case?
More information:
Number of OSDs: 5 physical server with 4x4TB - 16 GB of RAM - Core
i7 CPU
Number of Monitors: 1 virtual machine with 180 GB on SSD - 16 GB of
RAM - on an KVM Virtualization Machine
All Operating Systems: CentOS 7.2 with default kernel 3.10
All File Systems: XFS
Ceph Version: 10.2 Jewel
Switch for Private Networking: D-Link DGS-1008D Gigabit 8
NICs: Gb/s NIC x 2 for each server
Block Device on Client Server: Linux kernel RBD module
Thank you
Roozbeh
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
