On 16/06/17 09:48, Bharat Bhushan wrote:
> Hi Jean
>> +static int viommu_map(struct iommu_domain *domain, unsigned long iova,
>> + phys_addr_t paddr, size_t size, int prot) {
>> + int ret;
>> + struct viommu_domain *vdomain = to_viommu_domain(domain);
>> + struct virtio_iommu_req_map req = {
>> + .head.type = VIRTIO_IOMMU_T_MAP,
>> + .address_space = cpu_to_le32(vdomain->id),
>> + .virt_addr = cpu_to_le64(iova),
>> + .phys_addr = cpu_to_le64(paddr),
>> + .size = cpu_to_le64(size),
>> + };
>> +
>> + pr_debug("map %llu 0x%lx -> 0x%llx (%zu)\n", vdomain->id, iova,
>> + paddr, size);
>
> A query, when I am tracing above prints I see same physical address is mapped with two different virtual address, do you know why kernel does this?
That really depends which driver is calling into viommu. iommu_map is
called from the DMA API, which can be used by any device drivers. Within
an address space, multiple IOVAs pointing to the same PA isn't forbidden.
For example, looking at MAP requests for a virtio-net device, I get the
following trace:
ioas[1] map 0xfffffff3000 -> 0x8faa0000 (4096)
ioas[1] map 0xfffffff2000 -> 0x8faa0000 (4096)
ioas[1] map 0xfffffff1000 -> 0x8faa0000 (4096)
ioas[1] map 0xfffffff0000 -> 0x8faa0000 (4096)
ioas[1] map 0xffffffef000 -> 0x8faa0000 (4096)
ioas[1] map 0xffffffee000 -> 0x8faa0000 (4096)
ioas[1] map 0xffffffed000 -> 0x8faa0000 (4096)
ioas[1] map 0xffffffec000 -> 0x8faa0000 (4096)
ioas[1] map 0xffffffeb000 -> 0x8faa0000 (4096)
ioas[1] map 0xffffffea000 -> 0x8faa0000 (4096)
ioas[1] map 0xffffffe8000 -> 0x8faa0000 (8192)
...
During initialization, the virtio-net driver primes the rx queue with
receive buffers, that the host will then fill with network packets. It
calls virtqueue_add_inbuf_ctx to create descriptors on the rx virtqueue
for each buffer. Each buffer is 0x180 bytes here, so one 4k page can
contain around 10 (actually 11, with the last one crossing page boundary).
I guess the call trace goes like this:
virtnet_open
try_fill_recv
add_recvbuf_mergeable
virtqueue_add_inbuf_ctx
vring_map_one_sg
dma_map_page
__iommu_dma_map
But the IOMMU cannot map fragments of pages, since the granule is 0x1000.
Therefore when virtqueue_add_inbuf_ctx maps the buffer, __iommu_dma_map
aligns address and size on full pages. Someone motivated could probably
optimize this by caching mapped pages and reusing IOVAs, but currently
that's how it goes.
Thanks,
Jean
