On Wed, Oct 26, 2022 at 12:17 PM Stefano Garzarella <sgarzare@xxxxxxxxxx> wrote: > > On Wed, Oct 26, 2022 at 05:39:23PM +0800, Yongji Xie wrote: > >Hi Stefano, > > > >On Wed, Oct 26, 2022 at 5:12 PM Stefano Garzarella <sgarzare@xxxxxxxxxx> wrote: > >> > >> Hi Xie, > >> I was testing libblkio [1] with QSD vduse-blk export and had some > >> issues. > >> > >> In a nutshell, QSD prints me the following messages when using > >> vhost-vdpa to access the device: > >> > >> Failed to get vq[0] iova mapping > >> Failed to update vring for vq[0] > >> > >> This happens only with vhost-vdpa, using virtio-vdpa instead the device > >> works fine. > >> I'm using Linux v6.0 and QEMU master (commit > >> 214a8da23651f2472b296b3293e619fd58d9e212). > >> > >> I haven't had much time to investigate, I hope to do it next week, but > >> maybe it's much faster for you. > >> > >> I saw that ioctl(VDUSE_IOTLB_GET_FD) in libvduse.c returns -1 (EPERM), > >> so IIUC in the kernel vduse_dev_broken() was called, and the device is > >> in a broken state. > >> > >> > >> We will use libblkio in QEMU [2] to access vDPA devices via vhost-vdpa. > >> But I'm doing these tests without QEMU for now, using an example inside > >> the libblkio repo: > >> > >> # Build libblkio and examples > >> # Fedora/CentOS/RHEL > >> dnf install -y git meson rust cargo python3-docutils rustfmt > >> # Debian/Ubuntu > >> apt-get install -y git meson rustc cargo python3-docutils > >> > >> git clone https://gitlab.com/libblkio/libblkio.git > >> > >> cd libblkio > >> git checkout v1.1.0 > >> > >> meson setup build > >> meson compile -C build > >> > >> > >> # On terminal 1 > >> modprobe vduse > >> modprobe vhost-vdpa > >> > >> qemu-img create -f qcow2 -o preallocation=full /path/to/test.qcow2 1g > >> > >> qemu-storage-daemon \ > >> --blockdev file,filename=/path/to/test.qcow2,cache.direct=on,aio=native,node-name=file \ > >> --blockdev qcow2,file=file,node-name=qcow2 \ > >> --object iothread,id=iothread0 \ > >> --export vduse-blk,id=vduse0,name=vduse0,num-queues=1,node-name=qcow2,writable=on,iothread=iothread0 > >> > >> > >> # On terminal 2 > >> vdpa dev add name vduse0 mgmtdev vduse > >> > >> cd libblkio/build > >> > >> # blkio-bench executes > >> ./examples/blkio-bench virtio-blk-vhost-vdpa \ > >> path=/dev/vhost-vdpa-0 --runtime=5 --readwrite=randread > >> > >> # after this step, QSD (running on terminal 1) prints the following messages: > >> Failed to get vq[0] iova mapping > >> Failed to update vring for vq[0] > >> > >> I don't know if I'm doing something wrong or in libblkio we have some > >> issue, but using vdpa-sim-blk works correctly, so maybe there is > >> something in vduse that is missing. > >> > >> Any help or suggestion is welcome :-) > >> > > > >I'd like to know whether bio-bench uses the shared memory > >(tmpfs/hugetlbfs) as the vdpa memory region. This is what VDUSE needs. > > Okay, so IIUC every memory regions should have an associated fd. > > The buffers in libblkio are already allocated in this way, but it is not > true for the virtqueue memory, I'll change it and test. I just changed the virtqueue memory allocation as you suggested, and it worked! MR here: https://gitlab.com/libblkio/libblkio/-/merge_requests/141 Thanks for the help, Stefano _______________________________________________ Virtualization mailing list Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx https://lists.linuxfoundation.org/mailman/listinfo/virtualization
