On 2020/12/31 下午2:52, Yongji Xie wrote:
On Thu, Dec 31, 2020 at 1:50 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
On 2020/12/31 下午1:15, Yongji Xie wrote:
On Thu, Dec 31, 2020 at 10:49 AM Jason Wang <jasowang@xxxxxxxxxx> wrote:
On 2020/12/30 下午6:12, Yongji Xie wrote:
On Wed, Dec 30, 2020 at 4:41 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
On 2020/12/30 下午3:09, Yongji Xie wrote:
On Wed, Dec 30, 2020 at 2:11 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
On 2020/12/29 下午6:26, Yongji Xie wrote:
On Tue, Dec 29, 2020 at 5:11 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
----- Original Message -----
On Mon, Dec 28, 2020 at 4:43 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
On 2020/12/28 下午4:14, Yongji Xie wrote:
I see. So all the above two questions are because VHOST_IOTLB_INVALIDATE
is expected to be synchronous. This need to be solved by tweaking the
current VDUSE API or we can re-visit to go with descriptors relaying
first.
Actually all vdpa related operations are synchronous in current
implementation. The ops.set_map/dma_map/dma_unmap should not return
until the VDUSE_UPDATE_IOTLB/VDUSE_INVALIDATE_IOTLB message is replied
by userspace. Could it solve this problem?
I was thinking whether or not we need to generate IOTLB_INVALIDATE
message to VDUSE during dma_unmap (vduse_dev_unmap_page).
If we don't, we're probably fine.
It seems not feasible. This message will be also used in the
virtio-vdpa case to notify userspace to unmap some pages during
consistent dma unmapping. Maybe we can document it to make sure the
users can handle the message correctly.
Just to make sure I understand your point.
Do you mean you plan to notify the unmap of 1) streaming DMA or 2)
coherent DMA?
For 1) you probably need a workqueue to do that since dma unmap can
be done in irq or bh context. And if usrspace does't do the unmap, it
can still access the bounce buffer (if you don't zap pte)?
I plan to do it in the coherent DMA case.
Any reason for treating coherent DMA differently?
Now the memory of the bounce buffer is allocated page by page in the
page fault handler. So it can't be used in coherent DMA mapping case
which needs some memory with contiguous virtual addresses. I can use
vmalloc() to do allocation for the bounce buffer instead. But it might
cause some memory waste. Any suggestion?
I may miss something. But I don't see a relationship between the
IOTLB_UNMAP and vmalloc().
In the vmalloc() case, the coherent DMA page will be taken from the
memory allocated by vmalloc(). So IOTLB_UNMAP is not needed anymore
during coherent DMA unmapping because those vmalloc'ed memory which
has been mapped into userspace address space during initialization can
be reused. And userspace should not unmap the region until we destroy
the device.
Just to make sure I understand. My understanding is that IOTLB_UNMAP is
only needed when there's a change the mapping from IOVA to page.
Yes, that's true.
So if we stick to the mapping, e.g during dma_unmap, we just put IOVA to
free list to be used by the next IOVA allocating. IOTLB_UNMAP could be
avoided.
So we are not limited by how the pages are actually allocated?
In coherent DMA cases, we need to return some memory with contiguous
kernel virtual addresses. That is the reason why we need vmalloc()
here. If we allocate the memory page by page, the corresponding kernel
virtual addresses in a contiguous IOVA range might not be contiguous.
Yes, but we can do that as what has been done in the series
(alloc_pages_exact()). Or do you mean it would be a little bit hard to
recycle IOVA/pages here?
Yes, it might be hard to reuse the memory. For example, we firstly
allocate 1 IOVA/page during dma_map, then the IOVA is freed during
dma_unmap. Actually we can't reuse this single page if we need a
two-pages area in the next IOVA allocating. So the best way is using
IOTLB_UNMAP to free this single page during dma_unmap too.
Thanks,
Yongji
I get you now. Then I agree that let's go with IOTLB_UNMAP.
Thanks
