On Aug 11, 2014, at 8:35 PM, Christoffer Dall wrote:
> On Mon, Aug 11, 2014 at 12:24:35PM +0100, Richard W.M. Jones wrote:
>> On Mon, Aug 11, 2014 at 01:20:46PM +0200, Christoffer Dall wrote:
>>> On Sun, Aug 10, 2014 at 02:24:04PM +0100, Richard W.M. Jones wrote:
>>>> kvm_alloc_stage2_pgd has to do an order 9 allocation, ie. 512
>>>> contiguous pages I think.
>>>>
>>>> This often leads to problems running qemu when memory is relatively
>>>> low -- eg. if you have one VM running, a healthy number of host
>>>> applications, and perhaps "just" 4GB free; then you decide to run the
>>>> libguestfs test suite.
>>>>
>>>> Any suggestions how to deal with this?
>>>>
>>> I'm not familiar with the libguestfs test suite, but are you saying you
>>> have 4GB of free physical memory and when you start your first VM then
>>> you get this error? That sounds unlikely to me.
>>
>> No, it runs hundreds of appliances (not all at the same time). Some
>> fail.
>>
>> It seems to be a memory fragmentation issue, rather than the absolute
>> free memory.
>>
> Ok, that's what I thought. You can probably hack around it by reducing
> S2_PGD_ORDER to whatever is accessible by the VMs you wish to run (as
> Jungseok also points out), but I'm afraid an upstream solution is
> probably not ready before the next merge window opens, at least.
In case of ARM64 KVM, it is possible to reduce S2_PGD_ORDER in the following way.
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -62,7 +62,28 @@
* Align KVM with the kernel's view of physical memory. Should be
* 40bit IPA, with PGD being 8kB aligned in the 4KB page configuration.
*/
-#define KVM_PHYS_SHIFT PHYS_MASK_SHIFT
+static inline int kvm_get_pa_range(void)
+{
+ int pa_range = read_cpuid(ID_AA64MMFR0_EL1) & 0xf;
+
+ switch (pa_range) {
+ case 0:
+ return 32;
+ case 1:
+ return 36;
+ case 2:
+ return 40;
+ case 3:
+ return 42;
+ case 4:
+ return 44;
+ case 5:
+ return 48;
+ default:
+ return -EINVAL;
+ }
+}
+#define KVM_PHYS_SHIFT kvm_get_pa_range()
#define KVM_PHYS_SIZE (1UL << KVM_PHYS_SHIFT)
#define KVM_PHYS_MASK (KVM_PHYS_SIZE - 1UL)
The code puts limitation on guest's address space which is at most
host's physical address space. For example, if host runs on Cortex-A57,
IPA is set to 44, not 48.
If this approach looks reasonable, I will post it as 3.17-rc1 comes up.
If not, please ignore it or use it as hack.
- Jungseok Lee
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
