On Tue, 2012-07-24 at 13:20 -0700, Nicholas A. Bellinger wrote: > On Tue, 2012-07-24 at 09:57 +0200, Jan Kiszka wrote: > > On 2012-07-24 09:42, Nicholas A. Bellinger wrote: > > > Hi Anthony, Stefan & QEMU folks, > > > > > <SNIP> > > > > However, thus far I've not been able to get virtio-scsi <-> tcm_vhost > > > I/O to actually work against the latest qemu.git/master.. > > > > > > So while doing a (manual) bisection w/ this series to track down the > > > issue with qemu/master, I managed to run across something else.. With > > > the vhost-scsi series applied, everything is working as expected up > > > until the following commit: > > > > > > commit 1523ed9e1d46b0b54540049d491475ccac7e6421 > > > Author: Jan Kiszka <jan.kiszka@xxxxxxxxxxx> > > > Date: Thu May 17 10:32:39 2012 -0300 > > > > > > virtio/vhost: Add support for KVM in-kernel MSI injection > > > > > > > > > This commit ends up triggering the following assert immediately after > > > starting qemu with virtio-scsi <-> tcm_vhost: > > > > > > qemu-system-x86_64: /usr/src/qemu.git/hw/msix.c:515: > > > msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && > > > dev->msix_vector_release_notifier' failed. > > > > > > OK, so adding the following hack allows me to boot: <SNIP> > > Also, is there a git tree and a way to reproduce this without special > > hardware needs? > > > > I'll push this series + branches to demonstrate the issue into an public > tree this afternoon. > OK, the updated vhost-scsi QEMU working tree is now available here: http://git.kernel.org/?p=virt/kvm/nab/qemu-kvm.git;a=summary Please ignore the qemu-kvm.git name for the moment, as it really is based on upstream qemu. ;) The branch breakdown looks like: master 401a663 remove unused QemuOpts parameter from net init functions * vhost-scsi bdd00bd msix: Add msix_nr_vectors_allocated vhost-scsi-merge e4fdb8c [ahead 9] vhost-scsi: add -vhost-scsi host device vhost-scsi-workaround 32ed0db msix: Work-around for vhost-scsi with KVM in-kernel MSI injection - 'vhost-scsi' contains vhost-scsi patches + pre commit 1523ed9e1d logic (Running stable at ~67.5K IOPs, see below) - 'vhost-scsi-merge' contains a vhost-scsi series against yesterday's qemu.git HEAD (not working atm, but will be sending out for RFC shortly) - 'vhost-scsi-workaround' contains the msix.c workaround to get commit 1523ed9e1d to boot (reproducible performance regression, down to 6K IOPs) > Also, the particular backend is a Fusion-IO raw block flash device, but > I'm pretty sure that using a TCM RAMDISK into tcm_vhost would exhibit > the same type of behavior. (Will double check on that shortly..) > Interestingly enough, after running some test with the vhost-scsi branch above with the last good working commit (bdd00bd), the same fio benchmark inline below is now reaching ~67.5K IOPs using v3.5-rc2 guest + host.. rw=randrw rwmixwrite=25 rwmixread=75 size=32768m ioengine=libaio direct=1 iodepth=64 blocksize=4k numjobs=2 filename=/dev/sdb This ends up being within a few percentage of what the bare-metal HW is capable of.. (~70K IOPs per LUN for an ioDrive Duo, Jen's CC'ed). Note the older QEMU vhost-scsi tree based on pre 1.1.0 code was doing ~60K IOPs for the same test.. So this looks promising pre commit 1523ed9e1d vs. what we've been using with vhost-scsi so far.. ;) --nab -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
