On Wed, Oct 26, 2022 at 05:39:23PM +0800, Yongji Xie wrote:
Hi Stefano,
On Wed, Oct 26, 2022 at 5:12 PM Stefano Garzarella <sgarzare@xxxxxxxxxx> wrote:
Hi Xie,
I was testing libblkio [1] with QSD vduse-blk export and had some
issues.
In a nutshell, QSD prints me the following messages when using
vhost-vdpa to access the device:
Failed to get vq[0] iova mapping
Failed to update vring for vq[0]
This happens only with vhost-vdpa, using virtio-vdpa instead the device
works fine.
I'm using Linux v6.0 and QEMU master (commit
214a8da23651f2472b296b3293e619fd58d9e212).
I haven't had much time to investigate, I hope to do it next week, but
maybe it's much faster for you.
I saw that ioctl(VDUSE_IOTLB_GET_FD) in libvduse.c returns -1 (EPERM),
so IIUC in the kernel vduse_dev_broken() was called, and the device is
in a broken state.
We will use libblkio in QEMU [2] to access vDPA devices via vhost-vdpa.
But I'm doing these tests without QEMU for now, using an example inside
the libblkio repo:
# Build libblkio and examples
# Fedora/CentOS/RHEL
dnf install -y git meson rust cargo python3-docutils rustfmt
# Debian/Ubuntu
apt-get install -y git meson rustc cargo python3-docutils
git clone https://gitlab.com/libblkio/libblkio.git
cd libblkio
git checkout v1.1.0
meson setup build
meson compile -C build
# On terminal 1
modprobe vduse
modprobe vhost-vdpa
qemu-img create -f qcow2 -o preallocation=full /path/to/test.qcow2 1g
qemu-storage-daemon \
--blockdev file,filename=/path/to/test.qcow2,cache.direct=on,aio=native,node-name=file \
--blockdev qcow2,file=file,node-name=qcow2 \
--object iothread,id=iothread0 \
--export vduse-blk,id=vduse0,name=vduse0,num-queues=1,node-name=qcow2,writable=on,iothread=iothread0
# On terminal 2
vdpa dev add name vduse0 mgmtdev vduse
cd libblkio/build
# blkio-bench executes
./examples/blkio-bench virtio-blk-vhost-vdpa \
path=/dev/vhost-vdpa-0 --runtime=5 --readwrite=randread
# after this step, QSD (running on terminal 1) prints the following messages:
Failed to get vq[0] iova mapping
Failed to update vring for vq[0]
I don't know if I'm doing something wrong or in libblkio we have some
issue, but using vdpa-sim-blk works correctly, so maybe there is
something in vduse that is missing.
Any help or suggestion is welcome :-)
I'd like to know whether bio-bench uses the shared memory
(tmpfs/hugetlbfs) as the vdpa memory region. This is what VDUSE needs.
Okay, so IIUC every memory regions should have an associated fd.
The buffers in libblkio are already allocated in this way, but it is not
true for the virtqueue memory, I'll change it and test.
Thanks for the help :-)
Stefano
_______________________________________________
Virtualization mailing list
Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
