Bugs item #2113643, was opened at 2008-09-16 09:34 Message generated for change (Comment added) made by jessorensen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2113643&group_id=180599 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nikola Ciprich (nikola_ciprich) Assigned to: Nobody/Anonymous (nobody) Summary: guests AND host still getting stuck under CPU load Initial Comment: I'm experiencing problems with some KVM guests getting stuck for long time when all their "CPUs" are loaded. Host: 8xCPU intel, 2.6.26 x86_64, kvm-74, 12GB RAM Guest: 8XCPU, 2.6.24 x86_64 1GB RAM while doing kernel compilation on guest (using -j8), the guest sometimes hangs for long time, and on host I see: Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#3 stuck for 99s! [qemu-system-x86:17351] Sep 12 19:04:34 vbox1 [420515.344071] Modules linked in: tun bitrev drbd lock_dlm gfs2 cn crc32 dlm configfs ipmi_si ipmi_devintf ipmi_m sghandler ipt_REJECT xt_tcpudp xt_multiport nf_conntrack_ipv4 xt_state nf_conntrack iptable_filter ip_tables x_tables kvm_intel kvm ipv6 nfs lockd nfs_acl sunrpc 8021q bridge llc dm_mirror dm_log dm_mod wmi sbs sbshc fan battery backlight acpi_memhotplug ac lp nvram e1000 e sg snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss piix snd_pcm container pata_acpi ide_ pci_generic snd_timer snd soundcore snd_page_alloc parport_pc parport thermal processor ata_piix ata_generic button rng_core thermal_sys iTCO_wdt rtc_cmos rtc_core rtc_lib i5000_edac shpchp i2c_i801 i2c_core pci_hotplug edac_core pcspkr ide_disk ide_core aic94xx libsas fi rmware_class scsi_transport_sas ahci libata dock sd_mod scsi_mod raid456 async_xor async_memcpy async_tx xor raid1 ext3 jbd uhci_hcd ohc i_hcd ehci_hcd [last unloaded: scsi_wait_scan] Sep 12 19:04:34 vbox1 [420515.344071] CPU 3: Sep 12 19:04:34 vbox1 [420515.344071] Pid: 17351, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffff803e8cc1>] [<ffffffff803e8cc1>] sock_poll+0x11/0x20 Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff8100bd355a50 EFLAGS: 00000246 Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: 0000000000000080 RCX: ffffffff804c3520 Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: ffff8102632fe140 RDI: ffff8101b4c8bbc0 Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffff810217faa800 R08: 0000000000000001 R09: ffffffff8029e9df Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000000 R11: 00000000ffffffff R12: ffffffff8023e743 Sep 12 19:04:34 vbox1 [420515.344071] R13: 0000000000000001 R14: ffff8100bd355a08 R15: 00000000ffffffff Sep 12 19:04:34 vbox1 [420515.344071] FS: 00007f47f0a5e6f0(0000) GS:ffff81032fce65c0(0000) knlGS:0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00000000b7f43000 CR3: 00000000c01d6000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.344071] Sep 12 19:04:34 vbox1 [420515.344071] Call Trace: Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ac800>] ? do_select+0x330/0x590 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ad040>] ? __pollwait+0x0/0x130 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8022a440>] ? default_wake_function+0x0/0x10 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8022a440>] ? default_wake_function+0x0/0x10 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8022a440>] ? default_wake_function+0x0/0x10 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8022a440>] ? default_wake_function+0x0/0x10 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8022a440>] ? default_wake_function+0x0/0x10 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8022a440>] ? default_wake_function+0x0/0x10 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80249d99>] ? autoremove_wake_function+0x9/0x30 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80227cca>] ? __wake_up_common+0x5a/0x90 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80480a07>] ? _spin_lock_irqsave+0x37/0x50 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa05624dd>] ? :tun:tun_chr_aio_read+0x12d/0x3b0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802acca9>] ? core_sys_select+0x249/0x340 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80249d90>] ? autoremove_wake_function+0x0/0x30 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa03e7c6e>] ? :kvm:kvm_vm_ioctl+0x8e/0x270 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ad241>] ? sys_select+0xd1/0x1c0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#7 stuck for 99s! [syslog-ng:4636] Sep 12 19:04:34 vbox1 [420515.344071] Pid: 4636, comm: syslog-ng Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffff80480991>] [<ffffffff80480991>] _spin_unlock_irq+0x21/0x30 Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff81032d847e30 EFLAGS: 00000206 Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000007 RBX: 00000000000008ea RCX: 0000000000016dfa Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000001211 RSI: 0000000000000001 RDI: ffffffff805aedc0 Sep 12 19:04:34 vbox1 [420515.344071] RBP: 0000000000000001 R08: 0000000000000001 R09: ffffffff802355dc Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000008 R11: ffffffff803162e0 R12: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] R13: 0000000000000001 R14: 0000000000000003 R15: 0000000000000003 Sep 12 19:04:34 vbox1 [420515.344071] FS: 00007fcf4760a6e0(0000) GS:ffff81032fd8f340(0000) knlGS:0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00000000008b0000 CR3: 000000032dc88000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.344071] Sep 12 19:04:34 vbox1 [420515.344071] Call Trace: Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802355be>] ? do_syslog+0x10e/0x430 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8024c96e>] ? enqueue_hrtimer+0x6e/0xf0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802f1e40>] ? kmsg_read+0x0/0x60 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802f1e69>] ? kmsg_read+0x29/0x60 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802e82d1>] ? proc_reg_read+0x81/0xd0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8029da55>] ? vfs_read+0xc5/0x180 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8029df83>] ? sys_read+0x53/0x90 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:04:34 vbox1 [420515.344071] CPU 6: Sep 12 19:04:34 vbox1 [420515.344071] Pid: 9596, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffffa03ed750>] [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740 Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff810301937e18 EFLAGS: 00000206 Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: ffff810301dd4000 RCX: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: 0000000000000006 RDI: 0000000000004404 Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffffffffa040f125 R08: 00002b4835d83cc0 R09: 00002b4835d83cc0 Sep 12 19:04:34 vbox1 [420515.344071] R10: 00002b4835d83cc0 R11: ffffffff802540d0 R12: ffff81003f997810 Sep 12 19:04:34 vbox1 [420515.344071] R13: ffff81003f997810 R14: ffff810301dd4028 R15: ffff810301dd4028 Sep 12 19:04:34 vbox1 [420515.344071] FS: 0000000054379940(0000) GS:ffff81032fd8ff40(0000) knlGS:0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033 Sep 12 19:04:34 vbox1 [420515.344071] CR2: 000000000049e100 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.344071] Sep 12 19:04:34 vbox1 [420515.344071] Call Trace: Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#4 stuck for 100s! [qemu-system-x86:11474] Sep 12 19:04:34 vbox1 [420515.344071] Pid: 11474, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffffa03ed750>] [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740 Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff81025dc63e18 EFLAGS: 00000202 Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: ffff81025dc64000 RCX: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: 0000000080e4ece0 RDI: 0000000000004404 Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffffffffa040f125 R08: 0000000000000000 R09: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000000 R11: 0000000000000000 R12: 00000000fa723b58 Sep 12 19:04:34 vbox1 [420515.344071] R13: 00000000fa723b58 R14: 000000015dc64028 R15: ffff81025dc64028 Sep 12 19:04:34 vbox1 [420515.344071] FS: 0000000044f77940(0000) GS:ffff81032fcf28c0(0000) knlGS:0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00000000e1143000 CR3: 000000025e5eb000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.344071] Sep 12 19:04:34 vbox1 [420515.344071] Call Trace: Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80319d90>] ? selinux_file_ioctl+0x0/0xf0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:04:34 vbox1 [420515.344071] BUG: soft lockup - CPU#5 stuck for 100s! [qemu-system-x86:9591] Sep 12 19:04:34 vbox1 [420515.344071] Pid: 9591, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.344071] RIP: 0010:[<ffffffffa03ed750>] [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740 Sep 12 19:04:34 vbox1 [420515.344071] RSP: 0018:ffff8102f8c33e18 EFLAGS: 00000206 Sep 12 19:04:34 vbox1 [420515.344071] RAX: 0000000000000000 RBX: ffff8103185a8000 RCX: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000004404 Sep 12 19:04:34 vbox1 [420515.344071] RBP: ffffffffa040f125 R08: ffffffff80537e43 R09: 0000000000000001 Sep 12 19:04:34 vbox1 [420515.344071] R10: 0000000000000002 R11: 0000000000000706 R12: ffffffff80537d00 Sep 12 19:04:34 vbox1 [420515.344071] R13: ffffffff80537d00 R14: ffff8103185a8000 R15: ffff8103ff5fb380 Sep 12 19:04:34 vbox1 [420515.344071] FS: 0000000041312940(0000) GS:ffff81032fd7fc40(0000) knlGS:0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Sep 12 19:04:34 vbox1 [420515.344071] CR2: 00002ae6d1af26d0 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.344071] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.344071] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.344071] Sep 12 19:04:34 vbox1 [420515.344071] Call Trace: Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8029e9df>] ? fget_light+0x5f/0x100 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:04:34 vbox1 [420515.344071] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:04:34 vbox1 [420515.346213] BUG: soft lockup - CPU#0 stuck for 100s! [qemu-system-x86:9597] Sep 12 19:04:34 vbox1 [420515.346213] Pid: 9597, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.346213] RIP: 0010:[<ffffffff80212988>] [<ffffffff80212988>] native_read_tsc+0x8/0x20 Sep 12 19:04:34 vbox1 [420515.346213] RSP: 0018:ffff8102fad3dc80 EFLAGS: 00000202 Sep 12 19:04:34 vbox1 [420515.346213] RAX: 000000004e683734 RBX: 000000004e6836da RCX: 000000004e6836da Sep 12 19:04:34 vbox1 [420515.346213] RDX: 000000000000ff20 RSI: 0000000000000000 RDI: 0000000000000001 Sep 12 19:04:34 vbox1 [420515.346213] RBP: ffff8103238a23d8 R08: 0000000000000001 R09: ffffffffa03f3dbb Sep 12 19:04:34 vbox1 [420515.346213] R10: 0000000000000000 R11: ffffffffa04124b0 R12: 0000000000000007 Sep 12 19:04:34 vbox1 [420515.346213] R13: ffff8103238a23d8 R14: ffffffff8028292f R15: ffffe20010369e68 Sep 12 19:04:34 vbox1 [420515.346213] FS: 0000000054b7a940(0000) GS:ffffffff805e7000(0000) knlGS:ffff81003f99eb40 Sep 12 19:04:34 vbox1 [420515.346213] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033 Sep 12 19:04:34 vbox1 [420515.346213] CR2: 0000000000865e49 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.346213] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.346213] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.346213] Sep 12 19:04:34 vbox1 [420515.346213] Call Trace: Sep 12 19:04:34 vbox1 [420515.346213] [<ffffffff80342b94>] ? __delay+0x34/0x60 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffff80346ccd>] ? _raw_spin_lock+0x10d/0x180 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffffa03f3dbb>] ? :kvm:paging64_page_fault+0x15b/0x2b0 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffffa040f125>] ? :kvm_intel:vmcs_read32+0x5/0x10 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffffa0411f9d>] ? :kvm_intel:vmx_vcpu_run+0x20d/0x610 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffffa03f2939>] ? :kvm:kvm_mmu_page_fault+0x19/0x90 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffffa03ed7a2>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x2e2/0x740 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffffa03e863f>] ? :kvm:kvm_vcpu_ioctl+0x4df/0x580 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:04:34 vbox1 [420515.346225] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:04:34 vbox1 [420515.347070] BUG: soft lockup - CPU#2 stuck for 99s! [qemu-system-x86:9595] Sep 12 19:04:34 vbox1 [420515.347070] Pid: 9595, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.347070] RIP: 0010:[<ffffffffa03ed750>] [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740 Sep 12 19:04:34 vbox1 [420515.347070] RSP: 0018:ffff8102f65b1e18 EFLAGS: 00000206 Sep 12 19:04:34 vbox1 [420515.347070] RAX: 0000000000000000 RBX: ffff8102ff018000 RCX: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.347070] RDX: 0000000000000000 RSI: ffff81003e4b5288 RDI: 0000000000004404 Sep 12 19:04:34 vbox1 [420515.347070] RBP: ffffffffa040f125 R08: ffffffff80538044 R09: 0000000000000001 Sep 12 19:04:34 vbox1 [420515.347070] R10: 0000000000000002 R11: 00000000000001eb R12: ffff810001e35640 Sep 12 19:04:34 vbox1 [420515.347070] R13: ffff810001e35640 R14: ffff8102f65b1ce8 R15: ffffffffa03f26e0 Sep 12 19:04:34 vbox1 [420515.347070] FS: 0000000053b78940(0000) GS:ffff81032fc742c0(0000) knlGS:0000000000000000 Sep 12 19:04:34 vbox1 [420515.347070] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Sep 12 19:04:34 vbox1 [420515.347070] CR2: 00002b4a5bf2e6d0 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.347070] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.347070] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.347070] Sep 12 19:04:34 vbox1 [420515.347070] Call Trace: Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740 Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:04:34 vbox1 [420515.347070] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:04:34 vbox1 [420515.352399] BUG: soft lockup - CPU#1 stuck for 99s! [qemu-system-x86:9594] Sep 12 19:04:34 vbox1 [420515.352399] Pid: 9594, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:04:34 vbox1 [420515.352399] RIP: 0010:[<ffffffffa03ed750>] [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740 Sep 12 19:04:34 vbox1 [420515.352399] RSP: 0018:ffff8102f61b7e18 EFLAGS: 00000202 Sep 12 19:04:34 vbox1 [420515.352399] RAX: 0000000000000000 RBX: ffff8102f8570000 RCX: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.352399] RDX: 0000000000000000 RSI: ffff810001000000 RDI: 0000000000004404 Sep 12 19:04:34 vbox1 [420515.352399] RBP: ffffffffa040f125 R08: 0000000000000000 R09: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.352399] R10: 0000000000000001 R11: 0000000000000061 R12: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.352399] R13: 0000000000000000 R14: ffff8102f8570000 R15: ffff8102ff5fb380 Sep 12 19:04:34 vbox1 [420515.352399] FS: 0000000053377940(0000) GS:ffff81032fc74e40(0000) knlGS:0000000000000000 Sep 12 19:04:34 vbox1 [420515.352399] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033 Sep 12 19:04:34 vbox1 [420515.352399] CR2: 00002b6baae6d018 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:04:34 vbox1 [420515.352399] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:04:34 vbox1 [420515.352399] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:04:34 vbox1 [420515.352399] Sep 12 19:04:34 vbox1 [420515.352399] Call Trace: Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffffa03e8355>] ? :kvm:kvm_vcpu_ioctl+0x1f5/0x580 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffff8024d47c>] ? ktime_get+0xc/0x50 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:04:34 vbox1 [420515.352399] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:11:29 vbox1 [421081.190438] BUG: soft lockup - CPU#6 stuck for 77s! [qemu-system-x86:9597] Sep 12 19:11:29 vbox1 [421081.190438] Pid: 9597, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:11:29 vbox1 [421081.190438] RIP: 0033:[<0000000000513c79>] [<0000000000513c79>] Sep 12 19:11:29 vbox1 [421081.190438] RSP: 002b:0000000054b79fb0 EFLAGS: 00000217 Sep 12 19:11:29 vbox1 [421081.190438] RAX: 0000000000000000 RBX: 0000000002294330 RCX: ffffffffffffffff Sep 12 19:11:29 vbox1 [421081.190438] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000012 Sep 12 19:11:29 vbox1 [421081.190438] RBP: 0000000002294330 R08: 00000000007be8c0 R09: 0000009201ecae88 Sep 12 19:11:29 vbox1 [421081.190438] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000ba4728 Sep 12 19:11:29 vbox1 [421081.190438] R13: 0000000000000000 R14: 0000000054b7b000 R15: 0000000000001000 Sep 12 19:11:29 vbox1 [421081.190438] FS: 0000000054b7a940(0063) GS:ffff81032fd8ff40(0000) knlGS:0000000000000000 Sep 12 19:11:29 vbox1 [421081.190438] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033 Sep 12 19:11:29 vbox1 [421081.190438] CR2: 00002b9806261004 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:11:29 vbox1 [421081.190438] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.190438] DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400 Sep 12 19:11:29 vbox1 [421081.190438] Sep 12 19:11:29 vbox1 [421081.190438] Call Trace: Sep 12 19:11:29 vbox1 [421081.190438] ?!? EMPTY Sep 12 19:11:29 vbox1 [421081.190438] BUG: soft lockup - CPU#7 stuck for 78s! [ha_logger:30873] Sep 12 19:11:29 vbox1 [421081.190438] Pid: 30873, comm: ha_logger Not tainted 2.6.26lb.02 #1 Sep 12 19:11:29 vbox1 [421081.190438] RIP: 0010:[<ffffffffa001cc3b>] [<ffffffffa001cc3b>] :jbd:journal_dirty_metadata+0x4b/0x120 Sep 12 19:11:29 vbox1 [421081.190438] RSP: 0018:ffff8102a2c3bb68 EFLAGS: 00000207 Sep 12 19:11:29 vbox1 [421081.190438] RAX: 0000000000000001 RBX: ffff81032f8f2470 RCX: ffff81032dd90800 Sep 12 19:11:29 vbox1 [421081.190438] RDX: ffff81003cde2dc0 RSI: ffff81032f8f2470 RDI: ffff8101455d6438 Sep 12 19:11:29 vbox1 [421081.190438] RBP: ffff81032f8f2470 R08: 0000000000000000 R09: ffffffffa001babc Sep 12 19:11:29 vbox1 [421081.190438] R10: ffff81032d3c8000 R11: 0000000000000000 R12: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.190438] R13: ffff81032dd90b38 R14: ffffffffa001b1aa R15: ffff810100000007 Sep 12 19:11:29 vbox1 [421081.190438] FS: 0000000000000000(0000) GS:ffff81032fd8f340(0000) knlGS:0000000000000000 Sep 12 19:11:29 vbox1 [421081.190438] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Sep 12 19:11:29 vbox1 [421081.190438] CR2: 00007f94ced94190 CR3: 0000000309578000 CR4: 00000000000026e0 Sep 12 19:11:29 vbox1 [421081.190438] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.190438] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:11:29 vbox1 [421081.190438] Sep 12 19:11:29 vbox1 [421081.190438] Call Trace: Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffffa003d74c>] ? :ext3:__ext3_journal_dirty_metadata+0x2c/0x70 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffffa002ef58>] ? :ext3:ext3_mark_iloc_dirty+0x168/0x390 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffffa002f5aa>] ? :ext3:ext3_mark_inode_dirty+0x4a/0x60 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffffa0032a00>] ? :ext3:ext3_dirty_inode+0x90/0xb0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff802c04d4>] ? __mark_inode_dirty+0x34/0x1c0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff802b4aae>] ? touch_atime+0xfe/0x170 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff802a84ff>] ? __link_path_walk+0xbbf/0xeb0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff802a8844>] ? path_walk+0x54/0xb0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff802a8b2f>] ? do_path_lookup+0x8f/0x230 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff802a9afa>] ? __path_lookup_intent_open+0x6a/0xd0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff802a9f1e>] ? do_filp_open+0xbe/0x8d0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff80284cce>] ? handle_mm_fault+0x1ae/0x7e0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff8033fe72>] ? __down_read_trylock+0x42/0x60 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff80480677>] ? _spin_unlock+0x17/0x20 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff8029b186>] ? get_unused_fd_flags+0x106/0x130 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff8029b20a>] ? do_sys_open+0x5a/0xf0 Sep 12 19:11:29 vbox1 [421081.190438] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:11:29 vbox1 [421081.191044] BUG: soft lockup - CPU#0 stuck for 78s! [qemu-system-x86:9593] Sep 12 19:11:29 vbox1 [421081.191044] Pid: 9593, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:11:29 vbox1 [421081.191044] RIP: 0010:[<ffffffffa03ed750>] [<ffffffffa03ed750>] :kvm:kvm_arch_vcpu_ioctl_run+0x290/0x740 Sep 12 19:11:29 vbox1 [421081.191044] RSP: 0018:ffff8102fd891e18 EFLAGS: 00000206 Sep 12 19:11:29 vbox1 [421081.191044] RAX: 0000000000000000 RBX: ffff81031895a000 RCX: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.191044] RDX: 0000000000000000 RSI: 00000000006d8690 RDI: 0000000000004404 Sep 12 19:11:29 vbox1 [421081.191044] RBP: ffffffffa040f125 R08: 00007fffb70d6000 R09: 00007fffb70d5f60 Sep 12 19:11:29 vbox1 [421081.191044] R10: 0000000000000008 R11: 0000000000000206 R12: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.191044] R13: 0000000000000000 R14: ffff8102fd891ce8 R15: ffffffffa03f26e0 Sep 12 19:11:29 vbox1 [421081.191044] FS: 0000000052b76940(0000) GS:ffffffff805e7000(0000) knlGS:ffff81003f8776c0 Sep 12 19:11:29 vbox1 [421081.191044] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Sep 12 19:11:29 vbox1 [421081.191044] CR2: 00002b72f39e2810 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:11:29 vbox1 [421081.191044] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.191044] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:11:29 vbox1 [421081.191044] Sep 12 19:11:29 vbox1 [421081.191044] Call Trace: Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffffa03ed73e>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x27e/0x740 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffff8024d190>] ? hrtimer_start+0xe0/0x180 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:11:29 vbox1 [421081.191044] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:11:29 vbox1 [421081.191443] CPU 1: Sep 12 19:11:29 vbox1 [421081.191443] Pid: 9596, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:11:29 vbox1 [421081.191443] RIP: 0010:[<ffffffff80212988>] [<ffffffff80212988>] native_read_tsc+0x8/0x20 Sep 12 19:11:29 vbox1 [421081.191443] RSP: 0018:ffff810301937c80 EFLAGS: 00000202 Sep 12 19:11:29 vbox1 [421081.191443] RAX: 00000000dec0d1be RBX: 00000000dec0d164 RCX: 00000000dec0d164 Sep 12 19:11:29 vbox1 [421081.191443] RDX: 000000000000ffba RSI: 0000000000000000 RDI: 0000000000000001 Sep 12 19:11:29 vbox1 [421081.191443] RBP: ffff8103238a23d8 R08: 0000000000000001 R09: ffffffffa03f3dbb Sep 12 19:11:29 vbox1 [421081.191443] R10: 00002b67e706fa78 R11: ffffffffa04124b0 R12: 0000000000000007 Sep 12 19:11:29 vbox1 [421081.191443] R13: ffff8103238a23d8 R14: ffffffff8028292f R15: ffffe200113d10b0 Sep 12 19:11:29 vbox1 [421081.191443] FS: 0000000054379940(0000) GS:ffff81032fc74e40(0000) knlGS:ffff81003f8e0240 Sep 12 19:11:29 vbox1 [421081.191443] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Sep 12 19:11:29 vbox1 [421081.191443] CR2: 00002b67e7172000 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:11:29 vbox1 [421081.191443] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.191443] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:11:29 vbox1 [421081.191443] Sep 12 19:11:29 vbox1 [421081.191443] Call Trace: Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff80342b94>] ? __delay+0x34/0x60 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff80346ccd>] ? _raw_spin_lock+0x10d/0x180 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffffa03f3dbb>] ? :kvm:paging64_page_fault+0x15b/0x2b0 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffffa03f2600>] ? :kvm:paging32_shadow_walk_entry+0x150/0x230 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffffa040f125>] ? :kvm_intel:vmcs_read32+0x5/0x10 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffffa0411f9d>] ? :kvm_intel:vmx_vcpu_run+0x20d/0x610 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffffa03f2939>] ? :kvm:kvm_mmu_page_fault+0x19/0x90 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffffa03ed7a2>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x2e2/0x740 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff8024d47c>] ? ktime_get+0xc/0x50 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:11:29 vbox1 [421081.191443] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda Sep 12 19:11:29 vbox1 [421081.195429] BUG: soft lockup - CPU#3 stuck for 78s! [qemu-system-x86:9591] Sep 12 19:11:29 vbox1 [421081.195429] Pid: 9591, comm: qemu-system-x86 Not tainted 2.6.26lb.02 #1 Sep 12 19:11:29 vbox1 [421081.195429] RIP: 0010:[<ffffffff80212988>] [<ffffffff80212988>] native_read_tsc+0x8/0x20 Sep 12 19:11:29 vbox1 [421081.195429] RSP: 0018:ffff8102f8c33c80 EFLAGS: 00000206 Sep 12 19:11:29 vbox1 [421081.195429] RAX: 00000000decd0620 RBX: 0000000000000309 RCX: 00000000decd05b4 Sep 12 19:11:29 vbox1 [421081.195429] RDX: 000000000000ffba RSI: 0000000000000000 RDI: 0000000000000001 Sep 12 19:11:29 vbox1 [421081.195429] RBP: ffff8103238a23d8 R08: 0000000000000001 R09: ffffffffa03f3dbb Sep 12 19:11:29 vbox1 [421081.195429] R10: 0000000000000002 R11: ffffffffa04124b0 R12: 0000000000000007 Sep 12 19:11:29 vbox1 [421081.195429] R13: ffff8103238a23d8 R14: ffffffff8028292f R15: ffffe2000f9ecba0 Sep 12 19:11:29 vbox1 [421081.195429] FS: 0000000041312940(0000) GS:ffff81032fce65c0(0000) knlGS:0000000000000000 Sep 12 19:11:29 vbox1 [421081.195429] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b Sep 12 19:11:29 vbox1 [421081.195429] CR2: 00002b510b4526d0 CR3: 00000002f9106000 CR4: 00000000000026e0 Sep 12 19:11:29 vbox1 [421081.195429] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Sep 12 19:11:29 vbox1 [421081.195429] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Sep 12 19:11:29 vbox1 [421081.195429] Sep 12 19:11:29 vbox1 [421081.195429] Call Trace: Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff80342b75>] ? __delay+0x15/0x60 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff80346ccd>] ? _raw_spin_lock+0x10d/0x180 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa03f3dbb>] ? :kvm:paging64_page_fault+0x15b/0x2b0 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa03f4bd5>] ? :kvm:x86_decode_insn+0x6a5/0xc10 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff80480a07>] ? _spin_lock_irqsave+0x37/0x50 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa03f26e0>] ? :kvm:paging64_shadow_walk_entry+0x0/0x230 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa040f125>] ? :kvm_intel:vmcs_read32+0x5/0x10 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa0411f9d>] ? :kvm_intel:vmx_vcpu_run+0x20d/0x610 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa03f2939>] ? :kvm:kvm_mmu_page_fault+0x19/0x90 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa03ed7a2>] ? :kvm:kvm_arch_vcpu_ioctl_run+0x2e2/0x740 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff803160fb>] ? inode_has_perm+0x5b/0xa0 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffffa03e864d>] ? :kvm:kvm_vcpu_ioctl+0x4ed/0x580 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff8024d47c>] ? ktime_get+0xc/0x50 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff80316229>] ? file_has_perm+0xe9/0xf0 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff802ab60f>] ? vfs_ioctl+0x2f/0xa0 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff802ab6f4>] ? do_vfs_ioctl+0x74/0x2c0 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff802ab9d1>] ? sys_ioctl+0x91/0xb0 Sep 12 19:11:29 vbox1 [421081.195429] [<ffffffff8020b6c7>] ? tracesys+0xd5/0xda . . . etc During such hang, even the host gets unresponsive, it pings, but ssh stucks, etc. ---------------------------------------------------------------------- >Comment By: Jes Sorensen (jessorensen) Date: 2010-07-01 10:00 Message: Hi, Are either of you still seeing this problem? A lot of fixes have gone in since 2.6.27, and scalability should be a lot better in more recent versions. Thanks, Jes ---------------------------------------------------------------------- Comment By: Rafal Wijata (ravpl) Date: 2008-10-01 14:31 Message: It happens on my site as well. Lot's of (on the host, i killed the guest and all was gone) Oct 1 14:22:01 mailhub kernel: INFO: task imap:13290 blocked for more than 120 seconds. Oct 1 14:22:01 mailhub kernel: imap D ffff81002650eb04 0 13290 2728 Oct 1 14:22:01 mailhub kernel: ffff810219525e28 0000000000000082 ffff810219525d78 ffffffff810b4db4 Oct 1 14:22:01 mailhub kernel: ffff810219525e18 ffffffff81583700 ffffffff81583700 ffff81009b7516a0 Oct 1 14:22:01 mailhub kernel: ffff8102199f2d40 ffff81009b7519f0 000000051dbc5cc8 ffff81009b7519f0 Oct 1 14:22:01 mailhub kernel: Call Trace: Oct 1 14:22:01 mailhub kernel: [<ffffffff810b4db4>] ? vfs_permission+0x14/0x16 Oct 1 14:22:01 mailhub kernel: [<ffffffff810c2c99>] ? mntput_no_expire+0x2a/0x134 Oct 1 14:22:01 mailhub kernel: [<ffffffff8129b5f3>] __mutex_lock_slowpath+0x6b/0xa2 Oct 1 14:22:01 mailhub kernel: [<ffffffff8129b47e>] mutex_lock+0x2f/0x33 Oct 1 14:22:01 mailhub kernel: [<ffffffff810b696b>] do_unlinkat+0x78/0x16c Oct 1 14:22:01 mailhub kernel: [<ffffffff810af98d>] ? sys_newstat+0x31/0x3c Oct 1 14:22:01 mailhub kernel: [<ffffffff810b6a70>] sys_unlink+0x11/0x13 Oct 1 14:22:01 mailhub kernel: [<ffffffff8100c0ea>] system_call_after_swapgs+0x8a/0x8f This one is good, maybe i'm the winner here: 1674 seconds! Oct 1 14:22:01 mailhub kernel: BUG: soft lockup - CPU#7 stuck for 1674s! [swapper:0] Oct 1 14:22:01 mailhub kernel: [<ffffffff810b6a70>] sys_unlink+0x11/0x13 Oct 1 14:22:01 mailhub kernel: [<ffffffff8100c0ea>] system_call_after_swapgs+0x8a/0x8f Oct 1 14:22:01 mailhub kernel: Modules linked in: tun Oct 1 14:22:01 mailhub kernel: INFO: task imap:12581 blocked for more than 120 seconds. Oct 1 14:22:01 mailhub kernel: imap D ffff810116893e7c 0 12581 2728 Oct 1 14:22:01 mailhub kernel: ffff8101fc73fdf8 0000000000000086 ffff8101fc73fd68 ffffffff810b3da8 Oct 1 14:22:01 mailhub kernel: 00000000000041c0 ffffffff81583700 ffffffff81583700 ffff8101001bda80 Oct 1 14:22:01 mailhub kernel: ffff810218d6c3e0 ffff8101001bddd0 00000004fc73fe18 ffff8101001bddd0 Oct 1 14:22:01 mailhub kernel: Call Trace: Oct 1 14:22:01 mailhub kernel: kvm_intel kvm nfs [<ffffffff810b3da8>] ? permission+0x100/0x117 Oct 1 14:22:01 mailhub kernel: lockd nfs_acl autofs4 coretemp hwmon fuse sunrpc bridge ipt_REJECT nf_conntrack_ipv4 iptable_filter ip_tables xt_tcpudp nf_ conntrack_ipv6 xt_state nf_conntrack ip6t_REJECT ip6table_filter ip6_tables x_tables ipv6 cpufreq_ondemand acpi_cpufreq freq_table reiserfs ext2 dm_mirror dm_log dm_multipath dm_mod i2c_i801 i2c_core e1000e ata_piix iTCO_wdt iTCO_vendor_support pcspkr serio_raw pata_acpi ata_generic i5000_edac sg edac_core us b_storage ahci libata shpchp 3w_9xxx sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode] Oct 1 14:22:01 mailhub kernel: CPU 7: Oct 1 14:22:01 mailhub kernel: Modules linked in: tun kvm_intel kvm nfs lockd nfs_acl autofs4 coretemp hwmon fuse sunrpc bridge ipt_REJECT nf_conntrack_ip v4 iptable_filter ip_tables xt_tcpudp nf_conntrack_ipv6 xt_state nf_conntrack ip6t_REJECT ip6table_filter ip6_tables x_tables ipv6 cpufreq_ondemand acpi_cp ufreq freq_table reiserfs ext2 dm_mirror dm_log dm_multipath dm_mod i2c_i801 i2c_core e1000e ata_piix iTCO_wdt iTCO_vendor_support pcspkr serio_raw pata_ac pi ata_generic i5000_edac sg edac_core usb_storage ahci libata shpchp 3w_9xxx sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd [last unloaded: m icrocode] Oct 1 14:22:01 mailhub kernel: Pid: 0, comm: swapper Not tainted 2.6.26.3-29.fc9.x86_64 #1 Oct 1 14:22:01 mailhub kernel: RIP: 0010:[<ffffffff81012556>] [<ffffffff81012556>] mwait_idle+0x40/0x4c Oct 1 14:22:01 mailhub kernel: RSP: 0018:ffff81021f20ff08 EFLAGS: 00000246 Oct 1 14:22:01 mailhub kernel: [<ffffffff810c2c99>] ? mntput_no_expire+0x2a/0x134 Oct 1 14:22:01 mailhub kernel: RAX: 0000000000000000 RBX: ffff81021f20ff08 RCX: 0000000000000000 Oct 1 14:22:01 mailhub kernel: [<ffffffff8129b5f3>] __mutex_lock_slowpath+0x6b/0xa2 Oct 1 14:22:01 mailhub kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff813cfe30 Oct 1 14:22:01 mailhub kernel: [<ffffffff8129b47e>] mutex_lock+0x2f/0x33 Oct 1 14:22:01 mailhub kernel: RBP: ffff81000108d7c0 R08: 0000000000000000 R09: ffff810001043000 Oct 1 14:22:01 mailhub kernel: R10: ffff81000105aff0 R11: 0000000000000212 R12: ffffffff8104ce07 Oct 1 14:22:01 mailhub kernel: R13: ffff81021f20fe88 R14: 0000000088de1790 R15: ffff81021f20fe98 Oct 1 14:22:01 mailhub kernel: FS: 0000000000000000(0000) GS:ffff81021f081d80(0000) knlGS:0000000000000000 Oct 1 14:22:01 mailhub kernel: CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b Oct 1 14:22:01 mailhub kernel: CR2: 00007fe1cbf35000 CR3: 000000016e430000 CR4: 00000000000026e0 Oct 1 14:22:01 mailhub kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Oct 1 14:22:01 mailhub kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Oct 1 14:22:01 mailhub kernel: Call Trace: Oct 1 14:22:01 mailhub kernel: [<ffffffff8100b136>] ? enter_idle+0x22/0x24 Oct 1 14:22:01 mailhub kernel: [<ffffffff810b4771>] vfs_unlink+0x86/0xfc Oct 1 14:22:01 mailhub kernel: [<ffffffff810b69d1>] do_unlinkat+0xde/0x16c Oct 1 14:22:01 mailhub kernel: [<ffffffff8100b1ca>] ? cpu_idle+0x92/0xda Oct 1 14:22:01 mailhub kernel: [<ffffffff81296f69>] ? start_secondary+0x169/0x16d Oct 1 14:22:01 mailhub kernel: [<ffffffff810b86b0>] ? sys_fcntl+0x2f7/0x306 Oct 1 14:22:01 mailhub kernel: Oct 1 14:22:01 mailhub kernel: [<ffffffff810b6a70>] sys_unlink+0x11/0x13 Oct 1 14:22:01 mailhub kernel: [<ffffffff8100c0ea>] system_call_after_swapgs+0x8a/0x8f ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2113643&group_id=180599 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
