On 08/12/2011 04:27 PM, Liu Yuan wrote:
On 08/12/2011 12:50 PM, Badari Pulavarty wrote:
On 8/10/2011 8:19 PM, Liu Yuan wrote:
On 08/11/2011 11:01 AM, Liu Yuan wrote:
It looks like the patch wouldn't work for testing multiple devices.
vhost_blk_open() does
+ used_info_cachep = KMEM_CACHE(used_info, SLAB_HWCACHE_ALIGN |
SLAB_PANIC);
This is weird. how do you open multiple device?I just opened the
device with following command:
-drive file=/dev/sda6,if=virtio,cache=none,aio=native -drive
file=~/data0.img,if=virtio,cache=none,aio=native -drive
file=~/data1.img,if=virtio,cache=none,aio=native
And I didn't meet any problem.
this would tell qemu to open three devices, and pass three FDs to
three instances of vhost_blk module.
So KMEM_CACHE() is okay in vhost_blk_open().
Oh, you are right. KMEM_CACHE() is in the wrong place. it is three
instances vhost worker threads created. Hmmm, but I didn't meet any
problem when opening it and running it. So strange. I'll go to
figure it out.
When opening second device, we get panic since used_info_cachep is
already created. Just to make progress I moved this call to
vhost_blk_init().
I don't see any host panics now. With single block device (dd),
it seems to work fine. But when I start testing multiple block
devices I quickly run into hangs in the guest. I see following
messages in the guest from virtio_ring.c:
virtio_blk virtio2: requests: id 0 is not a head !
virtio_blk virtio1: requests: id 0 is not a head !
virtio_blk virtio4: requests: id 1 is not a head !
virtio_blk virtio3: requests: id 39 is not a head !
Thanks,
Badari
vq->data[] is initialized by guest virtio-blk driver and vhost_blk
is unware of it. it looks like used ID passed
over by vhost_blk to guest virtio_blk is wrong, but, it should not
happen. :|
And I can't reproduce this on my laptop. :(
Finally, found the issue :)
Culprit is:
+static struct io_event events[MAX_EVENTS];
With multiple devices, multiple threads could be executing
handle_completion() (one for
each fd) at the same time. "events" array is global :( Need to make
it one per device/fd.
For test, I changed MAX_EVENTS to 32 and moved "events" array to be
local (stack)
to handle_completion(). Tests are running fine.
Your laptop must have single processor, hence you have only one
thread executing handle_completion()
at any time..
Thanks,
Badari
Good catch, this is rather cool!....Yup, I develop it mostly in a
nested KVM environment. and the L2 host only runs single processor :(
Thanks,
Yuan
By the way, MAX_EVENTS should be 128, as much as guest virtio_blk driver
can batch-submit,
causing array overflow.
I have had turned on the debug, and had seen as much as over 100
requests batched from guest OS.
Thanks,
Yuan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
