https://bugzilla.kernel.org/show_bug.cgi?id=103851 Bug ID: 103851 Summary: qemu windows guest hangs on 100% cpu usage Product: Virtualization Version: unspecified Kernel Version: 3.13.6 Hardware: Intel OS: Linux Tree: Mainline Status: NEW Severity: blocking Priority: P1 Component: kvm Assignee: virtualization_kvm@xxxxxxxxxxxxxxxxxxxx Reporter: biaoxiangye@xxxxxxx Regression: No hi: I have two VM , one is winXP Prefessional SP3 32bit, another one is WindowsServer2008 Enterprise SP2 64bit. When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage. There are no message in kernel log when it happened, I try to rebuild kvm and add some debug info, I found the cpu exit reason is EXIT_REASON_PAUSE_INSTRUCTION. It seems like all the vcpu always in spinlock waiting. I'm not sure whether it's qemu's bug or kvm's. Any help would be appreciated. How reproducible: WinXP: seems always. WinServer2008: rare. Steps to Reproduce: winXP: hot reboot the xp guest os, hot reboot is necessary. WinServer2008: not sure, I didn't do anything, it just happened. The different between WinXP and WInServer2008: 1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine. 2. When WinServer2008 hangs, the vnc show the last screen and the screen won't change anything include system time. 3. When the VM hangs , if I execute "virsh suspend vm-name" and "virsh resume vm-name", the WinServer2008 will change to normal , and work fine not hangs anymore. But WinXP not change anything, still hangs. qemu version: QEMU emulator version 1.5.0, Copyright (c) 2003-2008 Fabrice Bellard host info: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz Ubuntu 12.04 LTS \n \l Linux cvknode2026 3.13.6 #1 SMP Fri Dec 12 09:17:35 CST 2014 x86_64 x86_64 x86_64 GNU/Linux qemu command line (guest OS XP): root 7124 1178 7.6 7750360 3761644 ? Sl 14:02 435:23 /usr/bin/kvm -name x -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 6144 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid d3832129-f77d-4b21-bbf7-fd337f53e572 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/x.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/vms/images/sn1-of-ff.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=0c:da:41:1d:f8:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/x.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:0 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 all qemu thread (guest OS XP): root@cvknode2026:/proc/7124/task# top -d 1 -H -p 7124 top - 14:37:05 up 7 days, 4:07, 1 user, load average: 10.71, 10.90, 10.19 Tasks: 14 total, 12 running, 2 sleeping, 0 stopped, 0 zombie Cpu(s): 38.8%us, 11.2%sy, 0.0%ni, 50.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 49159888k total, 35665128k used, 13494760k free, 436312k buffers Swap: 8803324k total, 0k used, 8803324k free, 28595100k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ P SWAP WCHAN COMMAND 7130 root 20 0 7568m 3.6g 6628 R 101 7.7 33:43.48 3 3.8g - kvm 7132 root 20 0 7568m 3.6g 6628 R 101 7.7 33:43.13 1 3.8g - kvm 7133 root 20 0 7568m 3.6g 6628 R 101 7.7 33:42.70 6 3.8g - kvm 7135 root 20 0 7568m 3.6g 6628 R 101 7.7 33:42.33 11 3.8g - kvm 7137 root 20 0 7568m 3.6g 6628 R 101 7.7 33:42.59 17 3.8g - kvm 7126 root 20 0 7568m 3.6g 6628 R 100 7.7 34:06.76 4 3.8g - kvm 7127 root 20 0 7568m 3.6g 6628 R 100 7.7 33:44.14 8 3.8g - kvm 7128 root 20 0 7568m 3.6g 6628 R 100 7.7 33:43.64 13 3.8g - kvm 7129 root 20 0 7568m 3.6g 6628 R 100 7.7 33:43.64 7 3.8g - kvm 7131 root 20 0 7568m 3.6g 6628 R 100 7.7 33:44.24 10 3.8g - kvm 7134 root 20 0 7568m 3.6g 6628 R 100 7.7 33:42.47 12 3.8g - kvm 7136 root 20 0 7568m 3.6g 6628 R 100 7.7 33:42.16 2 3.8g - kvm 7124 root 20 0 7568m 3.6g 6628 S 1 7.7 0:30.65 14 3.8g poll_sche kvm 7139 root 20 0 7568m 3.6g 6628 S 0 7.7 0:01.71 14 3.8g futex_wai kvm all thread's kernel stack (guest OS XP): root@cvknode2026:/proc/7124/task# cat 7130/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7132/stack [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7133/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7135/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffa02b6788>] vmx_vcpu_run+0x88/0x760 [kvm_intel] [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm] [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm] [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm] [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0 [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7137/stack [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7126/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7127/stack [<ffffffffa02b74f6>] handle_pause+0x16/0x30 [kvm_intel] [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7128/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7129/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7131/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7134/stack [<ffffffffa02b74fe>] handle_pause+0x1e/0x30 [kvm_intel] [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel] [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm] [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm] [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm] [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0 [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7136/stack [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7124/stack [<ffffffff811d50c9>] poll_schedule_timeout+0x49/0x70 [<ffffffff811d678a>] do_sys_poll+0x50a/0x590 [<ffffffff811d68eb>] SyS_poll+0x6b/0x100 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff root@cvknode2026:/proc/7124/task# cat 7139/stack [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150 [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0 [<ffffffff810ddc14>] do_futex+0x334/0xb70 [<ffffffff810de592>] SyS_futex+0x142/0x1a0 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff qemu command line (guest OS WinServer2008): root 25258 996 21.5 21174412 14181580 ? Sl Aug27 73740:11 /usr/bin/kvm -name zjx_1-clone -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 16384 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid 8c8b9abf-e9a6-4c3e-93cd-137a9550e593 -no-user-config -nodefaults -chardev so cket,id=charmonitor,path=/var/lib/libvirt/qemu/zjx_1-clone.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus =pci.0,addr=0x5 -drive file=/vms/aaa/zjx_1-clone.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=directsync -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/vms/isos/virtio-win2008R2.vfd,if=none,id=drive-fdc0-0-0,readonly=on,format=raw,cache=directsync -global isa-fdc.driveA=drive-fdc0-0-0 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0c:da:41:1d:b6:47,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-ser ial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/zjx_1-clone.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:3 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 all qemu thread (guest OS WinServer2008): top -d 1 -H -p 25258 top - 14:53:37 up 24 days, 21:27, 2 users, load average: 19.12, 20.56, 20.20 Tasks: 14 total, 13 running, 1 sleeping, 0 stopped, 0 zombie Cpu(s): 48.1%us, 18.2%sy, 0.0%ni, 33.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 65674944k total, 64651012k used, 1023932k free, 194608k buffers Swap: 8803324k total, 4140324k used, 4663000k free, 363712k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ P WCHAN COMMAND 25281 root 20 0 20.2g 13g 4020 R 157 21.6 5864:12 14 - kvm 25284 root 20 0 20.2g 13g 4020 R 155 21.6 5863:02 4 - kvm 25294 root 20 0 20.2g 13g 4020 R 153 21.6 5851:59 3 - kvm 25287 root 20 0 20.2g 13g 4020 R 152 21.6 5861:20 15 - kvm 25299 root 20 0 20.2g 13g 4020 R 152 21.6 5847:14 1 - kvm 25258 root 20 0 20.2g 13g 4020 R 122 21.6 3372:41 13 - kvm 25269 root 20 0 20.2g 13g 4020 R 101 21.6 5929:42 5 - kvm 25301 root 20 0 20.2g 13g 4020 R 101 21.6 5847:26 10 - kvm 25292 root 20 0 20.2g 13g 4020 R 100 21.6 5853:18 7 - kvm 25297 root 20 0 20.2g 13g 4020 R 100 21.6 5843:37 16 - kvm 25272 root 20 0 20.2g 13g 4020 R 98 21.6 5872:52 2 - kvm 25277 root 20 0 20.2g 13g 4020 R 93 21.6 5878:21 0 - kvm 25290 root 20 0 20.2g 13g 4020 R 51 21.6 5863:15 8 - kvm 25314 root 20 0 20.2g 13g 4020 S 0 21.6 0:41.42 1 futex_wai kvm all thread's kernel stack (guest OS WinServer2008): root@cvk11:/proc/25258/task# cat 25281/stack [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel] [<ffffffffa062cbb4>] __vcpu_run+0x704/0xc30 [kvm] [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm] [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm] [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0 [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25284/stack [<ffffffffa0613537>] kvm_vcpu_yield_to+0x47/0xa0 [kvm] [<ffffffffa06136ab>] kvm_vcpu_on_spin+0x11b/0x150 [kvm] [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25294/stack [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25287/stack [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25299/stack [<ffffffffa03d34f6>] handle_pause+0x16/0x30 [kvm_intel] [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel] [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm] [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm] [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm] [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0 [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25258/stack [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25269/stack [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel] [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel] [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm] [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm] [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm] [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0 [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25301/stack [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25292/stack [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25297/stack [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25272/stack [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25277/stack [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel] [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25290/stack [<ffffffffffffffff>] 0xffffffffffffffff root@cvk11:/proc/25258/task# cat 25314/stack [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150 [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0 [<ffffffff810ddc14>] do_futex+0x334/0xb70 [<ffffffff810de592>] SyS_futex+0x142/0x1a0 [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f [<ffffffffffffffff>] 0xffffffffffffffff -- You are receiving this mail because: You are watching the assignee of the bug. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
