Hello, On Tue, 8 Mar 2016 12:04:23 +0800 Pei Feng Lin wrote: > Hi, > > I setup a ceph environment with Hammer version. I run benchmark for the > rbd image with following command: > As always, exact versions of everything, kernel, OS, Ceph. Then your cluster, HW, network, how the client is connected, etc. > fio --ioengine=rbd --pool=performance_test --rbdname=test1 > --clientname=admin --iodepth=32 --direct=1 --rw=randwrite --bs=4k > --numjobs=4 --runtime=600 --ramp_time=100 --name=test --group_reporting > > You can see, 100s warm up and 600s testing ( --runtime=600 > --ramp_time=100 ) are using for the testing. At the first 300s testing, > the write IOPS is stable to 5000. But after 300s testing, the write IOPS > is not stable, sometimes the high IOPS is up to 4500, the low IOPS > drops down to 100. The IOPS jitter is very high, and is not stable. > There are a number of reasons this could be happening ranging from running out of page cache (your data no long fits in memory), journal flushes of the underlying filesystem and of course if you're using SSDs they could be going into garbage collection. One of the best tools to figure this out is running atop on your storage nodes. Christian > Does anyone meet the similar issue? How to tune the performance and make > it stable? > > Thanks! > > Peifeng Lin > > > -- Christian Balzer Network/Systems Engineer chibi@xxxxxxx Global OnLine Japan/Rakuten Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |