On Mon, 04 Oct 2021 11:30:06 +0100,
Lukas Jünger <lukas.juenger@xxxxxxxxxxxxxxxxxx> wrote:
>
> [1 <text/plain (en-US); utf-8 (quoted-printable)>]
> On 04.10.21 12:24, Marc Zyngier wrote:
> > Hi Lukas,
>
> Hi Mark,
>
> Thanks for your quick reply.
>
> > On Mon, 04 Oct 2021 11:07:47 +0100,
> > Lukas Jünger <lukas.juenger@xxxxxxxxxxxxxxxxxx> wrote:
> >> Hello,
> >>
> >> I am trying to run an emulator that uses KVM on arm64 to execute
> >> code. The emulator contains a userspace model of a GICv2 IRQ
> >> controller. The platform that I am running on (n1sdp) has a
> > N1-SDP? My condolences...
> Is there more to this?
How do you like the PCI patches? :D
> >> GICv3. When I boot Linux in the emulator I run into
> >> gic_check_cpu_features() in drivers/irqchip/irq-gic.c, which taints
> >> the kernel as the host uses system registers to communicate with the
> >> host GICv3. I saw that ICC_SRE_ELx can be used to force MMIO, but
> >> setting this from inside the VM did not work and using KVM_SET_ONE_REG
> >> failed with error.
> > N1-SDP doesn't implement the MMIO interface at all, and our GIC
> > emulation doesn't either. Both are valid implementations.
> >
> >> Is there a way to use a userspace GICv2 model with KVM on a GICv3 host
> >> without tainting?
> > The tainting happens because you have created a VM with a GICv3
> > irqchip (at some point, your VMM calls into KVM to create a device
> > with the KVM_DEV_TYPE_ARM_VGIC_V3 attribute). The guest then sees that
> > GICv3 is enabled (ICC_SRE_ELx.SRE==1), and yet you somehow expose a
> > GICv2 to the guest (either via DT or ACPI). That's illegal.
> >
> > If you want a userspace interrupt controller, you need prevent the
> > creation of an in-kernel interrupt controller, which is a change in
> > your VMM or maybe a configuration change.
> I'm not using an in-kernel irq controller, at least I don't set one
> up. This is all custom, so no QEMU etc. The GICv2 is also a custom
> model that lives in user space. The guest gets a DT telling it that
> there is a GICv2 and it should access it via MMIO. This all used to
> work on Raspberry Pi 3
RPI3 doesn't have a GIC at all, so the example is a bit moot.
> and Socionext Synquacer.
This one however is more interesting, as it has a GICv3 + v2 compat.
> The port to N1-SDP is
> giving me trouble. I understand why it is tainting the kernel, I was
> just wondering if I could somehow tell KVM to set this up correctly,
> e.g. by setting the ICC_SRE_ELx.
KVM doesn't *set* ICC_SRE_EL1.SRE. It is RAO/WI on this machine, which
is perfectly legal. However, KVM traps this access and emulates it
(access_gic_sre() returns vcpu->arch.vgic_cpu.vgic_v3.vgic_sre).
So if you see ICC_SRE_EL1.SRE==1 in your guest, that's because
vgic_sre is set to something that is non-zero. The only way for this
bit to be set is in vgic_v3_enable(), which has the following code:
<quote>
if (vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
vgic_v3->vgic_sre = (ICC_SRE_EL1_DIB |
ICC_SRE_EL1_DFB |
ICC_SRE_EL1_SRE);
vcpu->arch.vgic_cpu.pendbaser = INITIAL_PENDBASER_VALUE;
} else {
vgic_v3->vgic_sre = 0;
}
</quote>
So short of a terrible bug that would dump random values in this
structure, you are setting vgic_model to a GICv3 implementation. This
can only be done from userspace if you are creating a GICv3 irqchip.
Without seeing what your userspace does, I'm afraid I can't help you
much further. Can you please provide some traces of what it does? A
strace dump would certainly help.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
