On 07/15/2012 06:13 AM, Vladimir Bashkirtsev wrote:
Hello, Lately I was trying to get KVM to perform well on RBD. But it still appears elusive. [root@alpha etc]# rados -p rbd bench 120 seq -t 8 Total time run: 16.873277 Total reads made: 302 Read size: 4194304 Bandwidth (MB/sec): 71.592 Average Latency: 0.437984 Max latency: 3.26817 Min latency: 0.015786 Fairly good performance. But when I run in KVM: [root@mail ~]# hdparm -tT /dev/vda /dev/vda: Timing cached reads: 8808 MB in 2.00 seconds = 4411.49 MB/sec
This is just the guest page cache - it's reading the first two megabytes of the device repeatedly.
Timing buffered disk reads: 10 MB in 6.21 seconds = 1.61 MB/sec
This is a sequential read, so readahead in the guest should help here.
Not even close to what rados bench show! I even seen 900KB/sec performance. Such slow read performance of course affecting guests. Any ideas where to start to look for performance boost?
Do you have rbd caching enabled? It would also be interesting to see how the guest reads are translating to rados reads. hdparm is doing 2MiB sequential reads of the block device. If you add admin_socket=/var/run/ceph/kvm.asok to the rbd device on the qemu command line) you can see number of requests, latency, and request size info while the guest is running via: ceph --admin-daemon /var/run/ceph/kvm.asok perf dump Josh -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
