Hi,
For the device to work I need mem_type to continue to be PAGE_S2. But kvm_is_mmio_pfn() returns true as PageReserved() is true due to my memreserved region. I get an uncached device memory mapping as stage 2 overrides the mapping in stage 1 (and the cores do not see what the networking hardware wrote in the caches). If I instead remove the device memory completely from Linux, then pfn_valid() will be false and kvm_is_mmi_pfn() will still return true (but I would like all memory to be part of Linux, so not an idea I fancy). So what to do? Do I need to register the region differently with KVM (and Qemu), should I use the VFIO patches instead because they solve the problem, or does this code assume that all reserved pages are non-coherent device memory and it needs to be extended? Removing the code is of course not a solution even for me as there are plenty of other more normal devices on the SoC that are non-coherent.
--
I have a problem with the cacheability memory attribute permissions that are inserted into the stage 2 page table by KVM for ARM. Would be great if I could get some advise. Maybe I am doing something wrong in the set up or this kind of device has not been encountered before.
So here we go. I have a board with an LSI Axxia AXM5516 SoC used for networking equipment. In addition to 16 A15 cores, it contains a lot of networking HW. The networking HW communicates with the cores through the use of a part of the regular cache coherent DRAM memory. This region has to be consecutive and is today carved out using a memreserve in the device tree. Usually this region is several times larger than the amount of memory given to Linux.
My problem starts when I would like to access this region from a guest. I have made a direct mapping of this region into the guest by adding a sysbus_mmio region into the machine model of Qemu (which will end up calling the KVM_SET_USER_MEMORY_REGION ioctl). And yes I will start to use VFIO as soon as it is accepted ;-). When the stage 2 translation entry for this region is about to get entered in arch/arm/kvm/mmu.c:user_mem_abort() I get to this line:
if (kvm_is_mmio_pfn(pfn))
mem_type = PAGE_S2_DEVICE;
bool kvm_is_mmio_pfn(pfn_t pfn)
{
if (pfn_valid(pfn))
return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
return true;
}
mem_type = PAGE_S2_DEVICE;
bool kvm_is_mmio_pfn(pfn_t pfn)
{
if (pfn_valid(pfn))
return !is_zero_pfn(pfn) && PageReserved(pfn_to_page(pfn));
return true;
}
For the device to work I need mem_type to continue to be PAGE_S2. But kvm_is_mmio_pfn() returns true as PageReserved() is true due to my memreserved region. I get an uncached device memory mapping as stage 2 overrides the mapping in stage 1 (and the cores do not see what the networking hardware wrote in the caches). If I instead remove the device memory completely from Linux, then pfn_valid() will be false and kvm_is_mmi_pfn() will still return true (but I would like all memory to be part of Linux, so not an idea I fancy). So what to do? Do I need to register the region differently with KVM (and Qemu), should I use the VFIO patches instead because they solve the problem, or does this code assume that all reserved pages are non-coherent device memory and it needs to be extended? Removing the code is of course not a solution even for me as there are plenty of other more normal devices on the SoC that are non-coherent.
BTW, I cannot use the CMA (instead of memreserve) since it does not work with highmem.
Thanks: Magnus
Software Development Engineering Manager
Avago Technologies (formerly LSI Logic)
Box 1024, Knarrarnäsgatan 15
SE-164 21 Kista, Sweden
TEL +46 8 594 607 09
FAX +46 8 594 607 10
CELL +46 73 80 444 88
_______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
