On Mon, Jan 04, 2010 at 03:01:29PM +0100, Martin Schmitt wrote: > Ciao Marcelo, > > sorry for getting back so late. Thanks for your patience. :-) > > Marcelo Tosatti schrieb: > > >> I'm running a manually compiled KVM on CentOS 5.4. The KVM installation > >> has been carried over from CentOS 5.3, when KVM wasn't distributed with > >> the OS. (I tried to migrate to CentOS 5.4 native KVM support, but wasn't > >> able to get along with RedHat's interpretation of KVM.) > >> > >> The KVM version used is 88, on Kernel 2.6.18-128.7.1.el5, as KVM doesn't > >> seem to compile on CentOS' current 2.6.18-164.9.1.el5. > >> > >> Only on CentOS guests, I see very frequent "soft lockup" messages and > >> excessively hanging KVM instances. > > > > Can you please share some of the soft lockup messages. > > > > And how exactly are the VMs hanging? > > They are unresponsive for a few seconds. More "hiccuping" than hanging. > It appears to be I/O-related in some way, because it happens most > frequently when I do things on the file system. > > Dmesg is full of these: > > BUG: soft lockup - CPU#0 stuck for 10s! [kblockd/0:10] > > Pid: 10, comm: kblockd/0 > EIP: 0060:[<c056f931>] CPU: 0 > EIP is at ide_outb+0x4/0x5 > EFLAGS: 00000202 Not tainted (2.6.18-164.6.1.el5 #1) > EAX: 00000001 EBX: c07e2f80 ECX: 00000286 EDX: 0000c000 > ESI: 00000011 EDI: 00000000 EBP: c07e3014 DS: 007b ES: 007b > CR0: 8005003b CR2: b7f3c000 CR3: 12122000 CR4: 000006d0 > [<c0573cab>] ide_dma_start+0x22/0x2e > [<c0576474>] ide_do_rw_disk+0x3b2/0x4a6 > [<c056de34>] ide_do_request+0x533/0x6bf > [<c04de1b9>] freed_request+0x1d/0x37 > [<c056d8d0>] ide_end_request+0xcc/0xd4 > [<c056e221>] ide_intr+0x167/0x190 > [<c044da39>] handle_IRQ_event+0x45/0x8c > [<c044db04>] __do_IRQ+0x84/0xd6 > [<c044da80>] __do_IRQ+0x0/0xd6 > [<c04074b2>] do_IRQ+0x99/0xc3 > [<c0405946>] common_interrupt+0x1a/0x20 > [<c04291ab>] __do_softirq+0x57/0x114 > [<c04073cf>] do_softirq+0x52/0x9c > [<c04059d7>] apic_timer_interrupt+0x1f/0x24 > [<c056f931>] ide_outb+0x4/0x5 > [<c0573cab>] ide_dma_start+0x22/0x2e > [<c0576474>] ide_do_rw_disk+0x3b2/0x4a6 > [<c056de34>] ide_do_request+0x533/0x6bf > [<c04e710f>] cfq_kick_queue+0x70/0x80 > [<c0431e8a>] run_workqueue+0x78/0xb5 > [<c04e709f>] cfq_kick_queue+0x0/0x80 > [<c043273e>] worker_thread+0xd9/0x10b > [<c041e727>] default_wake_function+0x0/0xc > [<c0432665>] worker_thread+0x0/0x10b > [<c0434b55>] kthread+0xc0/0xeb > [<c0434a95>] kthread+0x0/0xeb > [<c0405c53>] kernel_thread_helper+0x7/0x10 > ======================= > >> by the hangs. The problem already was there on CentOS 5.3 as well. > >> With the Debian guests on the same host, I have never had any apparent > >> problems. > > > > Questions: > > > > - Is there significant swapping on the host? > > - Are you migrating vm's? > > No migration and no swap activity. The host has plenty of idle RAM: > > [root@zulu ~]# free -m > total used free shared buffers cached > Mem: 7987 7904 82 0 667 5101 > -/+ buffers/cache: 2135 5851 > Swap: 1983 0 1983 > > >> A number of google results suggest that I should work with CPU scaling > >> on the CentOS guest systems, but unfortunately, CPU scaling is not > >> available in my guests. So, here's my question: How do I enable CPU > >> scaling in KVM guests? Or is there any other measure against these soft > >> lockups that you can recommend? > > > > What probably was suggested is to disable cpu frequency scaling on the > > host. Please provide more details on the host system. > > Host is a Quadcore Xeon HP DL320 G5 with CentOS 5.4, old Kernel > 2.6.18-128.7.1.el5. > > There are no hints toward CPU scaling in /sys/devices/system/ on the host: > > [root@zulu ~]# ls -l /sys/devices/system/cpu/cpu0 > total 0 > drwxr-xr-x 5 root root 0 Nov 7 13:47 cache > -r-------- 1 root root 4096 Jan 4 14:55 crash_notes > drwxr-xr-x 2 root root 0 Nov 7 13:48 topology > > The file "Crash Notes" contains the following number: 22792b400 > > Thanks for your help, Martin, Can you please share a few more "soft lockup" messages? (with backtrace included). Also qemu command line. And boot-up messages of host and guest. Thanks -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
