Hi Russell,
Thanks for this.
On Fri, 27 Dec 2019 11:47:35 +0000,
Russell King <rmk+kernel@xxxxxxxxxxxxxxx> wrote:
>
> Booting 5.4 on LX2160A reveals that KVM is non-functional:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP intersecting with HYP VA, unable to continue
> kvm [1]: error initializing Hyp mode: -22
>
> Debugging shows:
>
> kvm [1]: IDMAP page: 81a26000
> kvm [1]: HYP VA range: 0:22ffffffff
>
> as RAM is located at:
>
> 80000000-fbdfffff : System RAM
> 2080000000-237fffffff : System RAM
Ouch. This looks like a terrible choice for a memory map.
>
> Comparing this with the same kernel on Armada 8040 shows:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 2a26000
> kvm [1]: HYP VA range: 4800000000:493fffffff
> ...
> kvm [1]: Hyp mode initialized successfully
>
> which indicates that hyp_va_msb is set, and is always set to the
> opposite value of the idmap page to avoid the overlap. This does not
> happen with the LX2160A.
>
> Further debugging shows vabits_actual = 39, kva_msb = 38 on LX2160A and
> kva_msb = 33 on Armada 8040. Looking at the bit layout of the HYP VA,
> there is still one bit available for hyp_va_msb. Set this bit
> appropriately. This allows kvm to be functional on the LX2160A, but
> without any HYP VA randomisation:
>
> kvm: Limiting the IPA size due to kernel Virtual Address limit
> kvm [1]: IPA Size Limit: 43bits
> kvm [1]: IDMAP page: 81a24000
> kvm [1]: HYP VA range: 4000000000:62ffffffff
> ...
> kvm [1]: Hyp mode initialized successfully
Nice bit of debugging. I guess part of the confusion is due to the
fact that the hyp_va_msb is part of the tag (as you found out), but
that the documentation doesn't really make that clear at all (and only
mentions the random part of the tag).
>
> Signed-off-by: Russell King <rmk+kernel@xxxxxxxxxxxxxxx>
> ---
> arch/arm64/kvm/va_layout.c | 22 +++++++++++++++-------
> 1 file changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c
> index 2cf7d4b606c3..83f8b3f51cf4 100644
> --- a/arch/arm64/kvm/va_layout.c
> +++ b/arch/arm64/kvm/va_layout.c
> @@ -22,6 +22,17 @@ static u8 tag_lsb;
> static u64 tag_val;
> static u64 va_mask;
>
> +/*
> + * We want to generate a hyp VA with the following format:
> + *
> + * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> + * ---------------------------------------------------------
> + * | 0000000 | hyp_va_msb | random tag | kern linear VA |
\---------- tag -------------/
How about the above, to make it clearer that the tag must include the
hyp_va_msb bit to avoid clashing with the IDMAP?
> + *
> + * which does not conflict with the idmap regions. This means that hyp_va_msb
> + * must always be present. Luckily, when kva_msb == (vabits_actual - 1) we
I'm not sure the "Luckily" part is appropriate here. Given the way
kva_msb is computed, I can't see how it can see how this can
fail.
This stems from the fact that the vabits space for kernel mappings is
split in two: vabits-1 for the linear map, and vabits-1 for the rest
(kernel text and co). Given that we define kva_msb as the highest
order bit that can change within the linear map, its value is at most
vabits-1.
> + * still have one bit for this, but no bits for the random tag.
> + */
> static void compute_layout(void)
> {
> phys_addr_t idmap_addr = __pa_symbol(__hyp_idmap_text_start);
> @@ -39,19 +50,16 @@ static void compute_layout(void)
> /*
> * No space in the address, let's compute the mask so
> * that it covers (vabits_actual - 1) bits, and the region
> - * bit. The tag stays set to zero.
> + * bit.
> */
> - va_mask = BIT(vabits_actual - 1) - 1;
> - va_mask |= hyp_va_msb;
> + tag_lsb = kva_msb;
> + va_mask = BIT(vabits_actual - 1) - 1;
> + tag_val = hyp_va_msb >> tag_lsb;
> } else {
> /*
> * We do have some free bits to insert a random tag.
> * Hyp VAs are now created from kernel linear map VAs
> * using the following formula (with V == vabits_actual):
> - *
> - * 63 ... V | V-1 | V-2 .. tag_lsb | tag_lsb - 1 .. 0
> - * ---------------------------------------------------------
> - * | 0000000 | hyp_va_msb | random tag | kern linear VA |
> */
> tag_lsb = kva_msb;
> va_mask = GENMASK_ULL(tag_lsb - 1, 0);
In the light of this, it'd be great to rework this code to simplify it
(getting rid of kva_msb, which really is tag_lsb) and make some of the
computing common to both branches.
Thanks,
M.
--
Jazz is not dead, it just smells funny.
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
