On Mon, 24 May 2021 15:40:22 +0100,
Andreas Färber <afaerber@xxxxxxx> wrote:
>
> On 17.05.21 11:29, Marc Zyngier wrote:
> > On Mon, 17 May 2021 00:05:47 +0100,
> > Andreas Färber <afaerber@xxxxxxx> wrote:
> >>
> >> Avoid the kernel getting stuck after:
> >>
> >> [ 1.175956] kvm [1]: IPA Size Limit: 40 bits
> >> [ 1.177164] kvm [1]: vgic-v2@ff320000
> >> [ 1.177545] kvm [1]: GIC system register CPU interface enabled
> >>
> >> or when dropping GICV reg entry:
> >>
> >> [ 1.176001] kvm [1]: IPA Size Limit: 40 bits
> >> [ 1.177191] kvm [1]: GICv3: no GICV resource entry
> >> [ 1.177664] kvm [1]: disabling GICv2 emulation
> >> [ 1.178115] kvm [1]: GIC system register CPU interface enabled
> >>
> >> Signed-off-by: Andreas Färber <afaerber@xxxxxxx>
> >> ---
> >> arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts | 4 ++++
> >> 1 file changed, 4 insertions(+)
> >>
> >> diff --git a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> >> index 2f8075d2391c..15293a8576c6 100644
> >> --- a/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> >> +++ b/arch/arm64/boot/dts/rockchip/rk1808k-toybrick-m0.dts
> >> @@ -48,6 +48,10 @@ &cpu1 {
> >> cpu-supply = <&vdd_cpu>;
> >> };
> >>
> >> +&gic {
> >> + /delete-property/ interrupts;
> >> +};
> >> +
> >> &uart2 {
> >> status = "okay";
> >> clocks = <&xin24m>;
> >
> > As I said in my reply to the cover letter, this is not an acceptable
> > outcome. Please add some debug to kvm_vgic_hyp_init() to understand
> > where this is hanging and why.
>
> Many thanks for that pointer.
>
> So, as alternative to dropping the DT interrupts property above, I could
> also work around this issue by commenting out
> vgic-init.c:vgic_init_cpu_starting()'s enable_percpu_irq() call.
>
> Otherwise I am seeing the following call flow:
>
> cpuhp_setup_state() -> __cpuhp_setup_state_cpuslocked() ->
> cpuhp_issue_call() -> cpuhp_invoke_ap_callback() -> __cpuhp_kick_ap() ->
> wait_for_ap_thread() -> wait_for_completion() --- doesn't return
>
> With kvm_info() / printk():
>
> [ 1.244079] kvm [1]: IPA Size Limit: 40 bits
>
> [ 1.245205] kvm [1]: vgic-v2@ff320000
>
> [ 1.245584] kvm [1]: GIC system register CPU interface enabled
>
> [ 1.246177] kvm [1]: before cpuhp_setup_state
>
> [ 1.246605] __cpuhp_setup_state_cpuslocked: kvm/arm/vgic:starting
>
> [ 1.247198] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
> state 225
>
> [ 1.247933] __cpuhp_setup_state_cpuslocked: for_each_present_cpu 0:
> before cpuhp_issue_call
>
> [ 1.248745] cpuhp_issue_call: before invoke
>
> [ 1.249154] cpuhp_issue_call: before AP invoke
>
> [ 1.249585] cpuhp_invoke_ap_callback
>
> [ 1.249936] cpuhp_invoke_ap_callback: after cpu_online
>
> [ 1.250435] cpuhp_invoke_ap_callback: before st->thread
>
> [ 1.250944] cpuhp_invoke_ap_callback: after st->thread
>
> [ 1.251445] __cpuhp_kick_ap
>
> [ 1.251731] __cpuhp_kick_ap: not returned
>
> [ 1.252140] vgic_init_cpu_starting: 9
>
> [ 1.252507] vgic_init_cpu_starting: done
>
> [ 1.255538] __cpuhp_kick_ap: wait_for_ap_thread
And you never see any RCU stall after that? It looks like a CPU has
disappeared in the weeds after enabling the per-CPU interrupt.
Please instrument what happens in
drivers/irqchip/irq-gic-v3.c::gic_unmask_irq() when d->hwirq == 9, a
well as vgic_maintenance_handler(), just in case it gets called...
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
