On 2011-07-24 18:11, Jan Kiszka wrote: >>> I had a closer look and identified two further issues, one generic, one >>> CPU-hotplug-specific: >>> - (qdev) devices that are hotplugged do not receive any reset. That >>> does not only apply to the APIC in case of CPU hotplugging, it is >>> also broken for NICs, storage controllers, etc. when doing PCI >>> hot-add as I just checked via gdb. >>> - CPU hotplugging was always (or at least for a fairly long time), >>> well, fragile as it failed to make CPU thread creation and CPU >>> initialization atomic against APIC addition and other initialization >>> steps. IOW, we need to create CPUs stopped, finish all init work, >>> sync their states completely to the kernel >>> (cpu_synchronize_post_init), and then kick them of. Actually I'm >> Syncing the state to the kernel should be done by vcpu thread, so I it >> cannot be stopped while the sync is done. May be I misunderstood what >> you mean here. > > Stopped first of all means not entering kvm_cpu_exec before the whole > setup and the initial sync are done. > > Syncing the initial state may also happen over the creating context as > long as the vcpus are stopped (analogously to > kvm_cpu_synchronize_post_init). OK, hacks below plus the following three patches make CPU hotplug work again - with some exceptions. Here are the patches: 1. http://thread.gmane.org/gmane.comp.emulators.kvm.devel/76484 2. http://thread.gmane.org/gmane.comp.emulators.qemu/110272 3. http://thread.gmane.org/gmane.comp.emulators.qemu/110426 And here are the hacks (well, the first hunk is clearly a fix, the last one clearly a hack, /me still undecided about the rest): diff --git a/hw/acpi_piix4.c b/hw/acpi_piix4.c index c30a050..f650250 100644 --- a/hw/acpi_piix4.c +++ b/hw/acpi_piix4.c @@ -92,7 +92,8 @@ static void pm_update_sci(PIIX4PMState *s) ACPI_BITMASK_POWER_BUTTON_ENABLE | ACPI_BITMASK_GLOBAL_LOCK_ENABLE | ACPI_BITMASK_TIMER_ENABLE)) != 0) || - (((s->gpe.sts[0] & s->gpe.en[0]) & PIIX4_PCI_HOTPLUG_STATUS) != 0); + (((s->gpe.sts[0] & s->gpe.en[0]) & + (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0); qemu_set_irq(s->irq, sci_level); /* schedule a timer interruption if needed */ diff --git a/hw/pc.c b/hw/pc.c index c0a88e1..e5371be 100644 --- a/hw/pc.c +++ b/hw/pc.c @@ -42,6 +42,7 @@ #include "kvm.h" #include "blockdev.h" #include "ui/qemu-spice.h" +#include "cpus.h" /* output Bochs bios info messages */ //#define DEBUG_BIOS @@ -936,6 +937,10 @@ CPUState *pc_new_cpu(const char *cpu_model) #endif } + if (vm_running) { + pause_all_vcpus(); + } + env = cpu_init(cpu_model); if (!env) { fprintf(stderr, "Unable to find x86 CPU definition\n"); @@ -947,6 +952,11 @@ CPUState *pc_new_cpu(const char *cpu_model) } qemu_register_reset(pc_cpu_reset, env); pc_cpu_reset(env); + + cpu_synchronize_post_init(env); + if (vm_running) { + resume_all_vcpus(); + } return env; } diff --git a/hw/qdev.c b/hw/qdev.c index 1626131..b91e2c2 100644 --- a/hw/qdev.c +++ b/hw/qdev.c @@ -330,6 +330,7 @@ BusState *sysbus_get_default(void) if (!main_system_bus) { main_system_bus = qbus_create(&system_bus_info, NULL, "main-system-bus"); + main_system_bus->allow_hotplug = 1; } return main_system_bus; } I see two remaining problems: - kvmclock is somehow broken, either in my guest kernel (OpenSUSE HEAD 3.0.0-2) or the host, -cpu host,-kvmclock works around sporadic guest lockups on echo 1 > /sys... - Seabios tends to lock up once every few system_reset after some CPU has been hot-added - also in TCG mode. It seems to dislike any setup of #CPUs > smp_cpus (whatever that implies in details). Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
