On 2011-07-25 15:21, Gleb Natapov wrote: > On Mon, Jul 25, 2011 at 03:18:19PM +0200, Jan Kiszka wrote: >> On 2011-07-24 18:11, Jan Kiszka wrote: >>>>> I had a closer look and identified two further issues, one generic, one >>>>> CPU-hotplug-specific: >>>>> - (qdev) devices that are hotplugged do not receive any reset. That >>>>> does not only apply to the APIC in case of CPU hotplugging, it is >>>>> also broken for NICs, storage controllers, etc. when doing PCI >>>>> hot-add as I just checked via gdb. >>>>> - CPU hotplugging was always (or at least for a fairly long time), >>>>> well, fragile as it failed to make CPU thread creation and CPU >>>>> initialization atomic against APIC addition and other initialization >>>>> steps. IOW, we need to create CPUs stopped, finish all init work, >>>>> sync their states completely to the kernel >>>>> (cpu_synchronize_post_init), and then kick them of. Actually I'm >>>> Syncing the state to the kernel should be done by vcpu thread, so I it >>>> cannot be stopped while the sync is done. May be I misunderstood what >>>> you mean here. >>> >>> Stopped first of all means not entering kvm_cpu_exec before the whole >>> setup and the initial sync are done. >>> >>> Syncing the initial state may also happen over the creating context as >>> long as the vcpus are stopped (analogously to >>> kvm_cpu_synchronize_post_init). >> >> OK, hacks below plus the following three patches make CPU hotplug work >> again - with some exceptions. Here are the patches: >> >> 1. http://thread.gmane.org/gmane.comp.emulators.kvm.devel/76484 >> 2. http://thread.gmane.org/gmane.comp.emulators.qemu/110272 >> 3. http://thread.gmane.org/gmane.comp.emulators.qemu/110426 >> >> And here are the hacks (well, the first hunk is clearly a fix, the last >> one clearly a hack, /me still undecided about the rest): >> >> diff --git a/hw/acpi_piix4.c b/hw/acpi_piix4.c >> index c30a050..f650250 100644 >> --- a/hw/acpi_piix4.c >> +++ b/hw/acpi_piix4.c >> @@ -92,7 +92,8 @@ static void pm_update_sci(PIIX4PMState *s) >> ACPI_BITMASK_POWER_BUTTON_ENABLE | >> ACPI_BITMASK_GLOBAL_LOCK_ENABLE | >> ACPI_BITMASK_TIMER_ENABLE)) != 0) || >> - (((s->gpe.sts[0] & s->gpe.en[0]) & PIIX4_PCI_HOTPLUG_STATUS) != 0); >> + (((s->gpe.sts[0] & s->gpe.en[0]) & >> + (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0); >> >> qemu_set_irq(s->irq, sci_level); >> /* schedule a timer interruption if needed */ >> diff --git a/hw/pc.c b/hw/pc.c >> index c0a88e1..e5371be 100644 >> --- a/hw/pc.c >> +++ b/hw/pc.c >> @@ -42,6 +42,7 @@ >> #include "kvm.h" >> #include "blockdev.h" >> #include "ui/qemu-spice.h" >> +#include "cpus.h" >> >> /* output Bochs bios info messages */ >> //#define DEBUG_BIOS >> @@ -936,6 +937,10 @@ CPUState *pc_new_cpu(const char *cpu_model) >> #endif >> } >> >> + if (vm_running) { >> + pause_all_vcpus(); >> + } >> + >> env = cpu_init(cpu_model); >> if (!env) { >> fprintf(stderr, "Unable to find x86 CPU definition\n"); >> @@ -947,6 +952,11 @@ CPUState *pc_new_cpu(const char *cpu_model) >> } >> qemu_register_reset(pc_cpu_reset, env); >> pc_cpu_reset(env); >> + >> + cpu_synchronize_post_init(env); >> + if (vm_running) { >> + resume_all_vcpus(); >> + } >> return env; >> } >> >> diff --git a/hw/qdev.c b/hw/qdev.c >> index 1626131..b91e2c2 100644 >> --- a/hw/qdev.c >> +++ b/hw/qdev.c >> @@ -330,6 +330,7 @@ BusState *sysbus_get_default(void) >> if (!main_system_bus) { >> main_system_bus = qbus_create(&system_bus_info, NULL, >> "main-system-bus"); >> + main_system_bus->allow_hotplug = 1; >> } >> return main_system_bus; >> } >> >> I see two remaining problems: >> - kvmclock is somehow broken, either in my guest kernel (OpenSUSE HEAD >> 3.0.0-2) or the host, -cpu host,-kvmclock works around sporadic >> guest lockups on echo 1 > /sys... >> - Seabios tends to lock up once every few system_reset after some >> CPU has been hot-added - also in TCG mode. It seems to dislike any >> setup of #CPUs > smp_cpus (whatever that implies in details). >> > Have you specified maxcpus? Something like -smp 1,maxcpus=4. Yes, for sure. BTW, cpu_set completely lacks any parameter sanity checks. That interface looks like "for gurus only". Hope we can do better via qdev and properties. Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
