[ kvm-Bugs-2113643 ] guests AND host still getting stuck under CPU load

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Bugs item #2113643, was opened at 2008-09-16 09:34
Message generated for change (Comment added) made by jessorensen
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2113643&group_id=180599

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Nikola Ciprich (nikola_ciprich)
Assigned to: Nobody/Anonymous (nobody)
Summary: guests AND host still getting stuck under CPU load

Initial Comment:
I'm experiencing problems with some KVM guests getting stuck for long time
when all their "CPUs" are loaded.
Host: 8xCPU intel, 2.6.26 x86_64, kvm-74, 12GB RAM
Guest: 8XCPU, 2.6.24 x86_64 1GB RAM

while doing kernel compilation on guest (using -j8), the guest sometimes hangs
for long time, and on host I see:
Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#3 stuck for 99s! [qemu-system-x86:17351]
Sep 12 19:04:34 vbox1 [420515.344071] Modules linked in: tun bitrev drbd lock_dlm gfs2 cn crc32 dlm configfs ipmi_si ipmi_devintf ipmi_m
sghandler ipt_REJECT xt_tcpudp xt_multiport nf_conntrack_ipv4 xt_state nf_conntrack iptable_filter ip_tables x_tables kvm_intel kvm ipv6
 nfs lockd nfs_acl sunrpc 8021q bridge llc dm_mirror dm_log dm_mod wmi sbs sbshc fan battery backlight acpi_memhotplug ac lp nvram e1000
e sg snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss piix snd_pcm container pata_acpi ide_
pci_generic snd_timer snd soundcore snd_page_alloc parport_pc parport thermal processor ata_piix ata_generic button rng_core thermal_sys
 iTCO_wdt rtc_cmos rtc_core rtc_lib i5000_edac shpchp i2c_i801 i2c_core pci_hotplug edac_core pcspkr ide_disk ide_core aic94xx libsas fi
rmware_class scsi_transport_sas ahci libata dock sd_mod scsi_mod raid456 async_xor async_memcpy async_tx xor raid1 ext3 jbd uhci_hcd ohc
i_hcd ehci_hcd [last unloaded: scsi_wait_scan]
Sep 12 19:04:34 vbox1 [420515.344071] CPU 3:
Sep 12 19:04:34 vbox1 [420515.344071] Pid: 17351, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffff803e8cc1>]  [<ffffffff803e8cc1>] sock_poll+0x11/0x20
Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff8100bd355a50  EFLAGS: 00000246
Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: 0000000000000080 RCX: ffffffff804c3520
Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: ffff8102632fe140 RDI: ffff8101b4c8bbc0
Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffff810217faa800 R08: 0000000000000001 R09: ffffffff8029e9df
Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000000 R11: 00000000ffffffff R12: ffffffff8023e743
Sep 12 19:04:34 vbox1 [420515.344071] R13: 0000000000000001 R14: ffff8100bd355a08 R15: 00000000ffffffff
Sep 12 19:04:34 vbox1 [420515.344071] FS:  00007f47f0a5e6f0(0000) GS:ffff81032fce65c0(0000) knlGS:0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00000000b7f43000 CR3: 00000000c01d6000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.344071]
Sep 12 19:04:34 vbox1 [420515.344071] Call Trace:
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ac800>] ? do_select+0x330/0x590
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ad040>] ? __pollwait+0x0/0x130
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8022a440>] ? default_wake_function+0x0/0x10
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8022a440>] ? default_wake_function+0x0/0x10
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8022a440>] ? default_wake_function+0x0/0x10
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8022a440>] ? default_wake_function+0x0/0x10
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8022a440>] ? default_wake_function+0x0/0x10
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8022a440>] ? default_wake_function+0x0/0x10
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80249d99>] ? autoremove_wake_function+0x9/0x30
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80227cca>] ? __wake_up_common+0x5a/0x90
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80480a07>] ? _spin_lock_irqsave+0x37/0x50
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa05624dd>] ? :tun:tun_chr_aio_read+0x12d/0x3b0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802acca9>] ? core_sys_select+0x249/0x340
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80249d90>] ? autoremove_wake_function+0x0/0x30
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa03e7c6e>] ? :kvm:kvm_vm_ioctl+0x8e/0x270
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ad241>] ? sys_select+0xd1/0x1c0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#7 stuck for 99s! [syslog-ng:4636]
Sep 12 19:04:34 vbox1 [420515.344071] Pid: 4636, comm: syslog-ng Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffff80480991>]  [<ffffffff80480991>] _spin_unlock_irq+0x21/0x30
Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff81032d847e30  EFLAGS: 00000206
Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000007 RBX: 00000000000008ea RCX: 0000000000016dfa
Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000001211 RSI: 0000000000000001 RDI: ffffffff805aedc0
Sep 12 19:04:34 vbox1 [420515.344071] RBP: 0000000000000001 R08: 0000000000000001 R09: ffffffff802355dc
Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000008 R11: ffffffff803162e0 R12: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] R13: 0000000000000001 R14: 0000000000000003 R15: 0000000000000003
Sep 12 19:04:34 vbox1 [420515.344071] FS:  00007fcf4760a6e0(0000) GS:ffff81032fd8f340(0000) knlGS:0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00000000008b0000 CR3: 000000032dc88000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.344071]
Sep 12 19:04:34 vbox1 [420515.344071] Call Trace:
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802355be>] ? do_syslog+0x10e/0x430
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8024c96e>] ? enqueue_hrtimer+0x6e/0xf0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802f1e40>] ? kmsg_read+0x0/0x60
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802f1e69>] ? kmsg_read+0x29/0x60
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802e82d1>] ? proc_reg_read+0x81/0xd0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8029da55>] ? vfs_read+0xc5/0x180
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8029df83>] ? sys_read+0x53/0x90
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

Sep 12 19:04:34 vbox1 [420515.344071] CPU 6:
Sep 12 19:04:34 vbox1 [420515.344071] Pid: 9596, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffffa03ed750>]  [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740
Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff810301937e18  EFLAGS: 00000206
Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: ffff810301dd4000 RCX: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: 0000000000000006 RDI: 0000000000004404
Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffffffffa040f125 R08: 00002b4835d83cc0 R09: 00002b4835d83cc0
Sep 12 19:04:34 vbox1 [420515.344071] R10: 00002b4835d83cc0 R11: ffffffff802540d0 R12: ffff81003f997810
Sep 12 19:04:34 vbox1 [420515.344071] R13: ffff81003f997810 R14: ffff810301dd4028 R15: ffff810301dd4028
Sep 12 19:04:34 vbox1 [420515.344071] FS:  0000000054379940(0000) GS:ffff81032fd8ff40(0000) knlGS:0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
Sep 12 19:04:34 vbox1 [420515.344071] CR2: 000000000049e100 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.344071]
Sep 12 19:04:34 vbox1 [420515.344071] Call Trace:
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#4 stuck for 100s! [qemu-system-x86:11474]
Sep 12 19:04:34 vbox1 [420515.344071] Pid: 11474, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffffa03ed750>]  [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740
Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff81025dc63e18  EFLAGS: 00000202
Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: ffff81025dc64000 RCX: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: 0000000080e4ece0 RDI: 0000000000004404
Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffffffffa040f125 R08: 0000000000000000 R09: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000000 R11: 0000000000000000 R12: 00000000fa723b58
Sep 12 19:04:34 vbox1 [420515.344071] R13: 00000000fa723b58 R14: 000000015dc64028 R15: ffff81025dc64028
Sep 12 19:04:34 vbox1 [420515.344071] FS:  0000000044f77940(0000) GS:ffff81032fcf28c0(0000) knlGS:0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00000000e1143000 CR3: 000000025e5eb000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.344071]
Sep 12 19:04:34 vbox1 [420515.344071] Call Trace:
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80319d90>] ? selinux_file_ioctl+0x0/0xf0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#5 stuck for 100s! [qemu-system-x86:9591]
Sep 12 19:04:34 vbox1 [420515.344071] Pid: 9591, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffffa03ed750>]  [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740
Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff8102f8c33e18  EFLAGS: 00000206
Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: ffff8103185a8000 RCX: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000004404
Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffffffffa040f125 R08: ffffffff80537e43 R09: 0000000000000001
Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000002 R11: 0000000000000706 R12: ffffffff80537d00
Sep 12 19:04:34 vbox1 [420515.344071] R13: ffffffff80537d00 R14: ffff8103185a8000 R15: ffff8103ff5fb380
Sep 12 19:04:34 vbox1 [420515.344071] FS:  0000000041312940(0000) GS:ffff81032fd7fc40(0000) knlGS:0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00002ae6d1af26d0 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.344071]
Sep 12 19:04:34 vbox1 [420515.344071] Call Trace:
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8029e9df>] ? fget_light+0x5f/0x100
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:04:34 vbox1 [420515.344071]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

Sep 12 19:04:34 vbox1 [420515.346213] BUG: soft lockup - CPU#0 stuck for 100s! [qemu-system-x86:9597]
Sep 12 19:04:34 vbox1 [420515.346213] Pid: 9597, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.346213] RIP: 0010:[<ffffffff80212988>]  [<ffffffff80212988>] native_read_tsc+0x8/0x20
Sep 12 19:04:34 vbox1 [420515.346213] RSP: 0018:ffff8102fad3dc80  EFLAGS: 00000202
Sep 12 19:04:34 vbox1 [420515.346213] RAX: 000000004e683734 RBX: 000000004e6836da RCX: 000000004e6836da
Sep 12 19:04:34 vbox1 [420515.346213] RDX: 000000000000ff20 RSI: 0000000000000000 RDI: 0000000000000001
Sep 12 19:04:34 vbox1 [420515.346213] RBP: ffff8103238a23d8 R08: 0000000000000001 R09: ffffffffa03f3dbb
Sep 12 19:04:34 vbox1 [420515.346213] R10: 0000000000000000 R11: ffffffffa04124b0 R12: 0000000000000007
Sep 12 19:04:34 vbox1 [420515.346213] R13: ffff8103238a23d8 R14: ffffffff8028292f R15: ffffe20010369e68
Sep 12 19:04:34 vbox1 [420515.346213] FS:  0000000054b7a940(0000) GS:ffffffff805e7000(0000) knlGS:ffff81003f99eb40
Sep 12 19:04:34 vbox1 [420515.346213] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
Sep 12 19:04:34 vbox1 [420515.346213] CR2: 0000000000865e49 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.346213] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.346213] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.346213]
Sep 12 19:04:34 vbox1 [420515.346213] Call Trace:
Sep 12 19:04:34 vbox1 [420515.346213]  [<ffffffff80342b94>] ? __delay+0x34/0x60
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffff80346ccd>] ? _raw_spin_lock+0x10d/0x180
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffffa03f3dbb>] ? :kvm:paging64_page_fault+0x15b/0x2b0
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffffa040f125>] ? :kvm_intel:vmcs_read32+0x5/0x10
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffffa0411f9d>] ? :kvm_intel:vmx_vcpu_run+0x20d/0x610
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffffa03f2939>] ? :kvm:kvm_mmu_page_fault+0x19/0x90
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffffa03ed7a2>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x2e2/0x740
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffffa03e863f>] ? :kvm:kvm_vcpu_ioctl+0x4df/0x580
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:04:34 vbox1 [420515.346225]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda
Sep 12 19:04:34 vbox1 [420515.347070] BUG: soft lockup - CPU#2 stuck for 99s! [qemu-system-x86:9595]
Sep 12 19:04:34 vbox1 [420515.347070] Pid: 9595, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.347070] RIP: 0010:[<ffffffffa03ed750>]  [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740
Sep 12 19:04:34 vbox1 [420515.347070] RSP: 0018:ffff8102f65b1e18  EFLAGS: 00000206
Sep 12 19:04:34 vbox1 [420515.347070] RAX: 0000000000000000 RBX: ffff8102ff018000 RCX: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.347070] RDX: 0000000000000000 RSI: ffff81003e4b5288 RDI: 0000000000004404
Sep 12 19:04:34 vbox1 [420515.347070] RBP: ffffffffa040f125 R08: ffffffff80538044 R09: 0000000000000001
Sep 12 19:04:34 vbox1 [420515.347070] R10: 0000000000000002 R11: 00000000000001eb R12: ffff810001e35640
Sep 12 19:04:34 vbox1 [420515.347070] R13: ffff810001e35640 R14: ffff8102f65b1ce8 R15: ffffffffa03f26e0
Sep 12 19:04:34 vbox1 [420515.347070] FS:  0000000053b78940(0000) GS:ffff81032fc742c0(0000) knlGS:0000000000000000
Sep 12 19:04:34 vbox1 [420515.347070] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
Sep 12 19:04:34 vbox1 [420515.347070] CR2: 00002b4a5bf2e6d0 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.347070] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.347070] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.347070]
Sep 12 19:04:34 vbox1 [420515.347070] Call Trace:
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:04:34 vbox1 [420515.347070]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda
Sep 12 19:04:34 vbox1 [420515.352399] BUG: soft lockup - CPU#1 stuck for 99s! [qemu-system-x86:9594]
Sep 12 19:04:34 vbox1 [420515.352399] Pid: 9594, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:04:34 vbox1 [420515.352399] RIP: 0010:[<ffffffffa03ed750>]  [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740
Sep 12 19:04:34 vbox1 [420515.352399] RSP: 0018:ffff8102f61b7e18  EFLAGS: 00000202
Sep 12 19:04:34 vbox1 [420515.352399] RAX: 0000000000000000 RBX: ffff8102f8570000 RCX: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.352399] RDX: 0000000000000000 RSI: ffff810001000000 RDI: 0000000000004404
Sep 12 19:04:34 vbox1 [420515.352399] RBP: ffffffffa040f125 R08: 0000000000000000 R09: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.352399] R10: 0000000000000001 R11: 0000000000000061 R12: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.352399] R13: 0000000000000000 R14: ffff8102f8570000 R15: ffff8102ff5fb380
Sep 12 19:04:34 vbox1 [420515.352399] FS:  0000000053377940(0000) GS:ffff81032fc74e40(0000) knlGS:0000000000000000
Sep 12 19:04:34 vbox1 [420515.352399] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
Sep 12 19:04:34 vbox1 [420515.352399] CR2: 00002b6baae6d018 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:04:34 vbox1 [420515.352399] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:04:34 vbox1 [420515.352399] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:04:34 vbox1 [420515.352399]
Sep 12 19:04:34 vbox1 [420515.352399] Call Trace:
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffffa03e8355>] ? :kvm:kvm_vcpu_ioctl+0x1f5/0x580
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffff8024d47c>] ? ktime_get+0xc/0x50
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:04:34 vbox1 [420515.352399]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

Sep 12 19:11:29 vbox1 [421081.190438] BUG: soft lockup - CPU#6 stuck for 77s! [qemu-system-x86:9597]
Sep 12 19:11:29 vbox1 [421081.190438] Pid: 9597, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:11:29 vbox1 [421081.190438] RIP: 0033:[<0000000000513c79>]  [<0000000000513c79>]
Sep 12 19:11:29 vbox1 [421081.190438] RSP: 002b:0000000054b79fb0  EFLAGS: 00000217
Sep 12 19:11:29 vbox1 [421081.190438] RAX: 0000000000000000 RBX: 0000000002294330 RCX: ffffffffffffffff
Sep 12 19:11:29 vbox1 [421081.190438] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000012
Sep 12 19:11:29 vbox1 [421081.190438] RBP: 0000000002294330 R08: 00000000007be8c0 R09: 0000009201ecae88
Sep 12 19:11:29 vbox1 [421081.190438] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000ba4728
Sep 12 19:11:29 vbox1 [421081.190438] R13: 0000000000000000 R14: 0000000054b7b000 R15: 0000000000001000
Sep 12 19:11:29 vbox1 [421081.190438] FS:  0000000054b7a940(0063) GS:ffff81032fd8ff40(0000) knlGS:0000000000000000
Sep 12 19:11:29 vbox1 [421081.190438] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
Sep 12 19:11:29 vbox1 [421081.190438] CR2: 00002b9806261004 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:11:29 vbox1 [421081.190438] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.190438] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Sep 12 19:11:29 vbox1 [421081.190438]
Sep 12 19:11:29 vbox1 [421081.190438] Call Trace:
Sep 12 19:11:29 vbox1 [421081.190438]
?!? EMPTY

Sep 12 19:11:29 vbox1 [421081.190438] BUG: soft lockup - CPU#7 stuck for 78s! [ha_logger:30873]
Sep 12 19:11:29 vbox1 [421081.190438] Pid: 30873, comm: ha_logger Not tainted 2.6.26lb.02 #1
Sep 12 19:11:29 vbox1 [421081.190438] RIP: 0010:[<ffffffffa001cc3b>]  [<ffffffffa001cc3b>] :jbd:journal_dirty_metadata+0x4b/0x120
Sep 12 19:11:29 vbox1 [421081.190438] RSP: 0018:ffff8102a2c3bb68  EFLAGS: 00000207
Sep 12 19:11:29 vbox1 [421081.190438] RAX: 0000000000000001 RBX: ffff81032f8f2470 RCX: ffff81032dd90800
Sep 12 19:11:29 vbox1 [421081.190438] RDX: ffff81003cde2dc0 RSI: ffff81032f8f2470 RDI: ffff8101455d6438
Sep 12 19:11:29 vbox1 [421081.190438] RBP: ffff81032f8f2470 R08: 0000000000000000 R09: ffffffffa001babc
Sep 12 19:11:29 vbox1 [421081.190438] R10: ffff81032d3c8000 R11: 0000000000000000 R12: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.190438] R13: ffff81032dd90b38 R14: ffffffffa001b1aa R15: ffff810100000007
Sep 12 19:11:29 vbox1 [421081.190438] FS:  0000000000000000(0000) GS:ffff81032fd8f340(0000) knlGS:0000000000000000
Sep 12 19:11:29 vbox1 [421081.190438] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Sep 12 19:11:29 vbox1 [421081.190438] CR2: 00007f94ced94190 CR3: 0000000309578000 CR4: 00000000000026e0
Sep 12 19:11:29 vbox1 [421081.190438] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.190438] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:11:29 vbox1 [421081.190438]
Sep 12 19:11:29 vbox1 [421081.190438] Call Trace:
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffffa003d74c>] ? :ext3:__ext3_journal_dirty_metadata+0x2c/0x70
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffffa002ef58>] ? :ext3:ext3_mark_iloc_dirty+0x168/0x390
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffffa002f5aa>] ? :ext3:ext3_mark_inode_dirty+0x4a/0x60
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffffa0032a00>] ? :ext3:ext3_dirty_inode+0x90/0xb0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff802c04d4>] ? __mark_inode_dirty+0x34/0x1c0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff802b4aae>] ? touch_atime+0xfe/0x170
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff802a84ff>] ? __link_path_walk+0xbbf/0xeb0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff802a8844>] ? path_walk+0x54/0xb0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff802a8b2f>] ? do_path_lookup+0x8f/0x230
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff802a9afa>] ? __path_lookup_intent_open+0x6a/0xd0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff802a9f1e>] ? do_filp_open+0xbe/0x8d0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff80284cce>] ? handle_mm_fault+0x1ae/0x7e0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff8033fe72>] ? __down_read_trylock+0x42/0x60
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff80480677>] ? _spin_unlock+0x17/0x20
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff8029b186>] ? get_unused_fd_flags+0x106/0x130
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff8029b20a>] ? do_sys_open+0x5a/0xf0
Sep 12 19:11:29 vbox1 [421081.190438]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda
Sep 12 19:11:29 vbox1 [421081.191044] BUG: soft lockup - CPU#0 stuck for 78s! [qemu-system-x86:9593]
Sep 12 19:11:29 vbox1 [421081.191044] Pid: 9593, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:11:29 vbox1 [421081.191044] RIP: 0010:[<ffffffffa03ed750>]  [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740
Sep 12 19:11:29 vbox1 [421081.191044] RSP: 0018:ffff8102fd891e18  EFLAGS: 00000206
Sep 12 19:11:29 vbox1 [421081.191044] RAX: 0000000000000000 RBX: ffff81031895a000 RCX: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.191044] RDX: 0000000000000000 RSI: 00000000006d8690 RDI: 0000000000004404
Sep 12 19:11:29 vbox1 [421081.191044] RBP: ffffffffa040f125 R08: 00007fffb70d6000 R09: 00007fffb70d5f60
Sep 12 19:11:29 vbox1 [421081.191044] R10: 0000000000000008 R11: 0000000000000206 R12: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.191044] R13: 0000000000000000 R14: ffff8102fd891ce8 R15: ffffffffa03f26e0
Sep 12 19:11:29 vbox1 [421081.191044] FS:  0000000052b76940(0000) GS:ffffffff805e7000(0000) knlGS:ffff81003f8776c0
Sep 12 19:11:29 vbox1 [421081.191044] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
Sep 12 19:11:29 vbox1 [421081.191044] CR2: 00002b72f39e2810 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:11:29 vbox1 [421081.191044] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.191044] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:11:29 vbox1 [421081.191044]
Sep 12 19:11:29 vbox1 [421081.191044] Call Trace:
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffff8024d190>] ? hrtimer_start+0xe0/0x180
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:11:29 vbox1 [421081.191044]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

Sep 12 19:11:29 vbox1 [421081.191443] CPU 1:
Sep 12 19:11:29 vbox1 [421081.191443] Pid: 9596, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:11:29 vbox1 [421081.191443] RIP: 0010:[<ffffffff80212988>]  [<ffffffff80212988>] native_read_tsc+0x8/0x20
Sep 12 19:11:29 vbox1 [421081.191443] RSP: 0018:ffff810301937c80  EFLAGS: 00000202
Sep 12 19:11:29 vbox1 [421081.191443] RAX: 00000000dec0d1be RBX: 00000000dec0d164 RCX: 00000000dec0d164
Sep 12 19:11:29 vbox1 [421081.191443] RDX: 000000000000ffba RSI: 0000000000000000 RDI: 0000000000000001
Sep 12 19:11:29 vbox1 [421081.191443] RBP: ffff8103238a23d8 R08: 0000000000000001 R09: ffffffffa03f3dbb
Sep 12 19:11:29 vbox1 [421081.191443] R10: 00002b67e706fa78 R11: ffffffffa04124b0 R12: 0000000000000007
Sep 12 19:11:29 vbox1 [421081.191443] R13: ffff8103238a23d8 R14: ffffffff8028292f R15: ffffe200113d10b0
Sep 12 19:11:29 vbox1 [421081.191443] FS:  0000000054379940(0000) GS:ffff81032fc74e40(0000) knlGS:ffff81003f8e0240
Sep 12 19:11:29 vbox1 [421081.191443] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
Sep 12 19:11:29 vbox1 [421081.191443] CR2: 00002b67e7172000 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:11:29 vbox1 [421081.191443] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.191443] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:11:29 vbox1 [421081.191443]
Sep 12 19:11:29 vbox1 [421081.191443] Call Trace:
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff80342b94>] ? __delay+0x34/0x60
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff80346ccd>] ? _raw_spin_lock+0x10d/0x180
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffffa03f3dbb>] ? :kvm:paging64_page_fault+0x15b/0x2b0
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffffa03f2600>] ? :kvm:paging32_shadow_walk_entry+0x150/0x230
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffffa040f125>] ? :kvm_intel:vmcs_read32+0x5/0x10
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffffa0411f9d>] ? :kvm_intel:vmx_vcpu_run+0x20d/0x610
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffffa03f2939>] ? :kvm:kvm_mmu_page_fault+0x19/0x90
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffffa03ed7a2>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x2e2/0x740
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff8024d47c>] ? ktime_get+0xc/0x50
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:11:29 vbox1 [421081.191443]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda
Sep 12 19:11:29 vbox1 [421081.195429] BUG: soft lockup - CPU#3 stuck for 78s! [qemu-system-x86:9591]
Sep 12 19:11:29 vbox1 [421081.195429] Pid: 9591, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1
Sep 12 19:11:29 vbox1 [421081.195429] RIP: 0010:[<ffffffff80212988>]  [<ffffffff80212988>] native_read_tsc+0x8/0x20
Sep 12 19:11:29 vbox1 [421081.195429] RSP: 0018:ffff8102f8c33c80  EFLAGS: 00000206
Sep 12 19:11:29 vbox1 [421081.195429] RAX: 00000000decd0620 RBX: 0000000000000309 RCX: 00000000decd05b4
Sep 12 19:11:29 vbox1 [421081.195429] RDX: 000000000000ffba RSI: 0000000000000000 RDI: 0000000000000001
Sep 12 19:11:29 vbox1 [421081.195429] RBP: ffff8103238a23d8 R08: 0000000000000001 R09: ffffffffa03f3dbb
Sep 12 19:11:29 vbox1 [421081.195429] R10: 0000000000000002 R11: ffffffffa04124b0 R12: 0000000000000007
Sep 12 19:11:29 vbox1 [421081.195429] R13: ffff8103238a23d8 R14: ffffffff8028292f R15: ffffe2000f9ecba0
Sep 12 19:11:29 vbox1 [421081.195429] FS:  0000000041312940(0000) GS:ffff81032fce65c0(0000) knlGS:0000000000000000
Sep 12 19:11:29 vbox1 [421081.195429] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
Sep 12 19:11:29 vbox1 [421081.195429] CR2: 00002b510b4526d0 CR3: 00000002f9106000 CR4: 00000000000026e0
Sep 12 19:11:29 vbox1 [421081.195429] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Sep 12 19:11:29 vbox1 [421081.195429] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Sep 12 19:11:29 vbox1 [421081.195429]
Sep 12 19:11:29 vbox1 [421081.195429] Call Trace:
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff80342b75>] ? __delay+0x15/0x60
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff80346ccd>] ? _raw_spin_lock+0x10d/0x180
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa03f3dbb>] ? :kvm:paging64_page_fault+0x15b/0x2b0
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa03f4bd5>] ? :kvm:x86_decode_insn+0x6a5/0xc10
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff80480a07>] ? _spin_lock_irqsave+0x37/0x50
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa03f26e0>] ? :kvm:paging64_shadow_walk_entry+0x0/0x230
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa040f125>] ? :kvm_intel:vmcs_read32+0x5/0x10
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa0411f9d>] ? :kvm_intel:vmx_vcpu_run+0x20d/0x610
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa03f2939>] ? :kvm:kvm_mmu_page_fault+0x19/0x90
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa03ed7a2>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x2e2/0x740
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff8024d47c>] ? ktime_get+0xc/0x50
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0
Sep 12 19:11:29 vbox1 [421081.195429]  [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda

.
.
.
etc
During such hang, even the host gets unresponsive, it pings, but ssh stucks, etc.



----------------------------------------------------------------------

>Comment By: Jes Sorensen (jessorensen)
Date: 2010-07-01 10:00

Message:
Hi,

Are either of you still seeing this problem? A lot of fixes have gone in
since 2.6.27, and scalability should be a lot better in more recent
versions.

Thanks,
Jes


----------------------------------------------------------------------

Comment By: Rafal Wijata (ravpl)
Date: 2008-10-01 14:31

Message:
It happens on my site as well. Lot's of (on the host, i killed the guest
and all was gone)

Oct  1 14:22:01 mailhub kernel: INFO: task imap:13290 blocked for more
than 120 seconds.
Oct  1 14:22:01 mailhub kernel: imap          D ffff81002650eb04     0
13290   2728
Oct  1 14:22:01 mailhub kernel: ffff810219525e28 0000000000000082
ffff810219525d78 ffffffff810b4db4
Oct  1 14:22:01 mailhub kernel: ffff810219525e18 ffffffff81583700
ffffffff81583700 ffff81009b7516a0
Oct  1 14:22:01 mailhub kernel: ffff8102199f2d40 ffff81009b7519f0
000000051dbc5cc8 ffff81009b7519f0
Oct  1 14:22:01 mailhub kernel: Call Trace:
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b4db4>] ?
vfs_permission+0x14/0x16
Oct  1 14:22:01 mailhub kernel: [<ffffffff810c2c99>] ?
mntput_no_expire+0x2a/0x134
Oct  1 14:22:01 mailhub kernel: [<ffffffff8129b5f3>]
__mutex_lock_slowpath+0x6b/0xa2
Oct  1 14:22:01 mailhub kernel: [<ffffffff8129b47e>] mutex_lock+0x2f/0x33
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b696b>]
do_unlinkat+0x78/0x16c
Oct  1 14:22:01 mailhub kernel: [<ffffffff810af98d>] ?
sys_newstat+0x31/0x3c
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b6a70>] sys_unlink+0x11/0x13
Oct  1 14:22:01 mailhub kernel: [<ffffffff8100c0ea>]
system_call_after_swapgs+0x8a/0x8f

This one is good, maybe i'm the winner here: 1674 seconds!

Oct  1 14:22:01 mailhub kernel: BUG: soft lockup - CPU#7 stuck for 1674s!
[swapper:0]
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b6a70>] sys_unlink+0x11/0x13
Oct  1 14:22:01 mailhub kernel: [<ffffffff8100c0ea>]
system_call_after_swapgs+0x8a/0x8f
Oct  1 14:22:01 mailhub kernel: Modules linked in: tun
Oct  1 14:22:01 mailhub kernel: INFO: task imap:12581 blocked for more
than 120 seconds.
Oct  1 14:22:01 mailhub kernel: imap          D ffff810116893e7c     0
12581   2728
Oct  1 14:22:01 mailhub kernel: ffff8101fc73fdf8 0000000000000086
ffff8101fc73fd68 ffffffff810b3da8
Oct  1 14:22:01 mailhub kernel: 00000000000041c0 ffffffff81583700
ffffffff81583700 ffff8101001bda80
Oct  1 14:22:01 mailhub kernel: ffff810218d6c3e0 ffff8101001bddd0
00000004fc73fe18 ffff8101001bddd0
Oct  1 14:22:01 mailhub kernel: Call Trace:
Oct  1 14:22:01 mailhub kernel: kvm_intel kvm nfs [<ffffffff810b3da8>] ?
permission+0x100/0x117
Oct  1 14:22:01 mailhub kernel: lockd nfs_acl autofs4 coretemp hwmon fuse
sunrpc bridge ipt_REJECT nf_conntrack_ipv4 iptable_filter ip_tables
xt_tcpudp nf_
conntrack_ipv6 xt_state nf_conntrack ip6t_REJECT ip6table_filter
ip6_tables x_tables ipv6 cpufreq_ondemand acpi_cpufreq freq_table reiserfs
ext2 dm_mirror
dm_log dm_multipath dm_mod i2c_i801 i2c_core e1000e ata_piix iTCO_wdt
iTCO_vendor_support pcspkr serio_raw pata_acpi ata_generic i5000_edac sg
edac_core us
b_storage ahci libata shpchp 3w_9xxx sd_mod scsi_mod ext3 jbd mbcache
uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Oct  1 14:22:01 mailhub kernel: CPU 7:
Oct  1 14:22:01 mailhub kernel: Modules linked in: tun kvm_intel kvm nfs
lockd nfs_acl autofs4 coretemp hwmon fuse sunrpc bridge ipt_REJECT
nf_conntrack_ip
v4 iptable_filter ip_tables xt_tcpudp nf_conntrack_ipv6 xt_state
nf_conntrack ip6t_REJECT ip6table_filter ip6_tables x_tables ipv6
cpufreq_ondemand acpi_cp
ufreq freq_table reiserfs ext2 dm_mirror dm_log dm_multipath dm_mod
i2c_i801 i2c_core e1000e ata_piix iTCO_wdt iTCO_vendor_support pcspkr
serio_raw pata_ac
pi ata_generic i5000_edac sg edac_core usb_storage ahci libata shpchp
3w_9xxx sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd [last
unloaded: m
icrocode]
Oct  1 14:22:01 mailhub kernel: Pid: 0, comm: swapper Not tainted
2.6.26.3-29.fc9.x86_64 #1
Oct  1 14:22:01 mailhub kernel: RIP: 0010:[<ffffffff81012556>] 
[<ffffffff81012556>] mwait_idle+0x40/0x4c
Oct  1 14:22:01 mailhub kernel: RSP: 0018:ffff81021f20ff08  EFLAGS:
00000246
Oct  1 14:22:01 mailhub kernel: [<ffffffff810c2c99>] ?
mntput_no_expire+0x2a/0x134
Oct  1 14:22:01 mailhub kernel: RAX: 0000000000000000 RBX:
ffff81021f20ff08 RCX: 0000000000000000
Oct  1 14:22:01 mailhub kernel: [<ffffffff8129b5f3>]
__mutex_lock_slowpath+0x6b/0xa2
Oct  1 14:22:01 mailhub kernel: RDX: 0000000000000000 RSI:
0000000000000001 RDI: ffffffff813cfe30
Oct  1 14:22:01 mailhub kernel: [<ffffffff8129b47e>] mutex_lock+0x2f/0x33
Oct  1 14:22:01 mailhub kernel: RBP: ffff81000108d7c0 R08:
0000000000000000 R09: ffff810001043000
Oct  1 14:22:01 mailhub kernel: R10: ffff81000105aff0 R11:
0000000000000212 R12: ffffffff8104ce07
Oct  1 14:22:01 mailhub kernel: R13: ffff81021f20fe88 R14:
0000000088de1790 R15: ffff81021f20fe98
Oct  1 14:22:01 mailhub kernel: FS:  0000000000000000(0000)
GS:ffff81021f081d80(0000) knlGS:0000000000000000
Oct  1 14:22:01 mailhub kernel: CS:  0010 DS: 0018 ES: 0018 CR0:
000000008005003b
Oct  1 14:22:01 mailhub kernel: CR2: 00007fe1cbf35000 CR3:
000000016e430000 CR4: 00000000000026e0
Oct  1 14:22:01 mailhub kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Oct  1 14:22:01 mailhub kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Oct  1 14:22:01 mailhub kernel: Call Trace:
Oct  1 14:22:01 mailhub kernel: [<ffffffff8100b136>] ?
enter_idle+0x22/0x24
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b4771>] vfs_unlink+0x86/0xfc
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b69d1>]
do_unlinkat+0xde/0x16c
Oct  1 14:22:01 mailhub kernel: [<ffffffff8100b1ca>] ? cpu_idle+0x92/0xda
Oct  1 14:22:01 mailhub kernel: [<ffffffff81296f69>] ?
start_secondary+0x169/0x16d
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b86b0>] ?
sys_fcntl+0x2f7/0x306
Oct  1 14:22:01 mailhub kernel:
Oct  1 14:22:01 mailhub kernel: [<ffffffff810b6a70>] sys_unlink+0x11/0x13
Oct  1 14:22:01 mailhub kernel: [<ffffffff8100c0ea>]
system_call_after_swapgs+0x8a/0x8f


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2113643&group_id=180599
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux