On Tue, Dec 22, 2020 at 08:29:20PM +0800, Jason Wang wrote:
On 2020/12/22 下午6:57, Stefano Garzarella wrote:
On Tue, Dec 22, 2020 at 10:44:48AM +0800, Jason Wang wrote:
On 2020/12/21 下午7:14, Stefano Garzarella wrote:
On Mon, Dec 21, 2020 at 11:16:54AM +0800, Jason Wang wrote:
On 2020/12/18 下午7:38, Stefano Garzarella wrote:
On Mon, Nov 16, 2020 at 11:37:48AM +0800, Jason Wang wrote:
On 2020/11/13 下午9:47, Stefano Garzarella wrote:
Thanks to Max that started this work!
I took his patches, and extended the block simulator a bit.
This series moves the network device simulator in a new module
(vdpa_sim_net) and leaves the generic functions in the
vdpa_sim core
module, allowing the possibility to add new vDPA device simulators.
Then we added a new vdpa_sim_blk module to simulate a block device.
I'm not sure about patch 11 ("vringh: allow
vringh_iov_xfer() to skip
bytes when ptr is NULL"), maybe we can add a new
functions instead of
modify vringh_iov_xfer().
As Max reported, I'm also seeing errors with
vdpa_sim_blk related to
iotlb and vringh when there is high load, these are some
of the error
messages I can see randomly:
vringh: Failed to access avail idx at 00000000e8deb2cc
vringh: Failed to read head: idx 6289 address 00000000e1ad1d50
vringh: Failed to get flags at 000000006635d7a3
virtio_vdpa vdpa0: vringh_iov_push_iotlb() error: -14
offset: 0x2840000 len: 0x20000
virtio_vdpa vdpa0: vringh_iov_pull_iotlb() error: -14
offset: 0x58ee000 len: 0x3000
These errors should all be related to the fact that
iotlb_translate()
fails with -EINVAL, so it seems that we miss some mapping.
Is this only reproducible when there's multiple co-current
accessing of IOTLB? If yes, it's probably a hint that some
kind of synchronization is still missed somewhere.
It might be useful to log the dma_map/unmp in both
virtio_ring and vringh to see who is missing the map.
Just an update about these issues with vdpa-sim-blk.
I've been focusing a little bit on these failures over the
last few days and have found two issues related to the
IOTLB/IOMMU:
1. Some requests coming from the block layer fills the SG
list with multiple buffers that had the same physical
address. This happens for example while using 'mkfs', at
some points multiple sectors are zeroed so multiple SG
elements point to the same physical page that is zeroed.
Since we are using vhost_iotlb_del_range() in the
vdpasim_unmap_page(), this removes all the overlapped
ranges. I fixed removing a single map in
vdpasim_unmap_page(), but has an alternative we can
implement some kind of reference counts.
I think we need to do what hardware do. So using refcount is
probably not a good ida.
Okay, so since we are using for simplicity an identical mapping,
we are assigning the same dma_addr to multiple pages.
I think I get you now. That's the root cause for the failure.
Yes, sorry, I didn't explain well previously.
Then I think we need an simple iova allocator for vdpa simulator,
and it might be useful for VDUSE as well.
Okay, I'll work on it.
If you have an example to follow or some pointers, they are welcome :-)
Kernel had implemented one in iova.c but I'm not sure we need the
complexity like that. Or we can just use rbtree or idr to implement a
simpler one.
Yeah, I found it and I started to integrate it in the simulator.
Also if it appears complicated, it seams to me that it should be simple
to integrate.
I'll give it a try, and if it is too complicate, I'll switch to a simple
rbtree.
Thanks,
Stefano
_______________________________________________
Virtualization mailing list
Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
