On Mon, Jan 16, 2017 at 8:15 AM, Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote: > On Tue, Jan 03, 2017 at 12:02:14PM -0500, Weiwei Jia wrote: >> > The expensive part is the virtqueue kick. Recently we tried polling the >> > virtqueue instead of waiting for the ioeventfd file descriptor and got >> > double-digit performance improvements: >> > https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00148.html >> > >> > If you want to understand the performance of your benchmark you'll have >> > compare host/guest disk stats (e.g. request lifetime, disk utilization, >> > queue depth, average request size) to check that the bare metal and >> > guest workloads are really sending comparable I/O patterns to the >> > physical disk. >> > >> > Then you using Linux and/or QEMU tracing to analyze the request latency >> > by looking at interesting points in the request lifecycle like virtqueue >> > kick, host Linux AIO io_submit(2), etc. >> > >> >> Thank you. I will look into "polling the virtqueue" as you said above. >> Currently, I just use blktrace to see disk stats and add logs in the >> I/O workload to see the time latency for each request. What kind of >> tools are you using to analyze request lifecycle like virtqueue kick, >> host Linux AIO iosubmit, etc. >> >> Do you trace the lifecycle like this >> (http://www.linux-kvm.org/page/Virtio/Block/Latency#Performance_data) >> but it seems to be out of date. Does it >> (http://repo.or.cz/qemu-kvm/stefanha.git/shortlog/refs/heads/tracing-dev-0.12.4) >> still work on QEMU 2.4.1? > > The details are out of date but the general approach to tracing the I/O > request lifecycle still apply. > > There are multiple tracing tools that can do what you need. I've CCed > Karl Rister who did the latest virtio-blk dataplane tracing. > > "perf record -a -e kvm:\*" is a good start. You can use "perf probe" to > trace QEMU's trace events (recent versions have sdt support, which means > SystemTap tracepoints work) and also trace any function in QEMU: > http://blog.vmsplice.net/2011/03/how-to-use-perf-probe.html Thank you. I will try it. Best, Weiwei Jia -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
