Hi Takahiro, On Mon, 2015-02-02 at 17:18 +0900, AKASHI Takahiro wrote: > On 01/31/2015 04:48 AM, Geoff Levand wrote: > > As an initial implementation we can hook into the CPU_DYING_FROZEN > > notifier sent to hyp_init_cpu_notify(). The longer term solution > > should use kvm_arch_hardware_enable() and kvm_arch_hardware_disable(). > > Are these two different approaches? Yes, these are two different solutions, One initial work-around, and a more involved proper solution. Hooking into the CPU_DYING_FROZEN notifier would be a initial fix. The proper solution would be to move the KVM setup to kvm_arch_hardware_enable(), and the shutdown to kvm_arch_hardware_disable(). > kernel_exec() -> machine_shutdown() -> disable_nonboot_cpu() > -> _cpu_down() -> cpu_notify_nofail(CPU_DEAD|...) > > On the other hand, kvm already has a hook into kvm_arch_hardware_disable(): > (ignoring kvm_usage_count here) > kvm_cpu_hotplug(CPU_DYING) -> hardware_disable() > -> hardware_disable_nolock() -> kvm_arch_hardware_disable() > > So it seems that we don't have to add a new hook at hyp_init_cpu_notify() > if kvm_arch_hardware_disable() is properly implemented. Yes, that is correct. But, as above, you would also need to update the KVM startup to use kvm_arch_hardware_enable(). > disable_nonboot_cpu() will not inovke cpu hotplug on *boot* cpu, and > we should handle it in a separate way though. IIRC, the secondary cpus go through PSCI on shutdown, and that path is working OK. Maybe I am mistaken though. The primary cpu shutdown (hyp stubs restored) is what is missing. The primary cpu goes through cpu_soft_restart(), and that is what is currently failing. -Geoff