On Wed, Mar 26, 2014 at 8:45 AM, Raphael Bauduin <rblists@xxxxxxxxx> wrote:
Hi,Can this be due to memory over-commitment even if some memory is still used by the kernel for caches and buffers? (collectd graph shows no free memory, with 15G used, very little buffers, and 1G cache). There are 32GB of swap, of which only 150MB are used.
we have regular crashed of a kvm host with the error "unable to handle paging request".
I suspect might be the direction to search to find the cause, but would be happy to learn from people versed in the kernel behaviour to confirm or reject my hypothesis. Below is the full error.
Thanks!
Raph
745 Mar 23 14:27:37 sMaster01 kernel: [241450.355339] BUG: unable to handle kernel paging request at ffff8804c001fade
746 Mar 23 14:27:37 sMaster01 kernel: [241450.355384] IP: [<ffffffff8117e9e9>] bio_check_eod+0x29/0xcd
747 Mar 23 14:27:37 sMaster01 kernel: [241450.355433] PGD 1002063 PUD 0
748 Mar 23 14:27:37 sMaster01 kernel: [241450.355464] Oops: 0000 [#1] SMP
749 Mar 23 14:27:37 sMaster01 kernel: [241450.355496] last sysfs file: /sys/devices/system/cpu/cpu15/topology/thread_siblings
750 Mar 23 14:27:37 sMaster01 kernel: [241450.355551] CPU 4
751 Mar 23 14:27:37 sMaster01 kernel: [241450.355577] Modules linked in: ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT xt_tcpudp kvm_amd kvm ip6table_filter ip6_tables iptable_fi lter ip_tables x_tables tun nfsd exportfs nfs lockd fscache nfs_acl auth_rpcgss sunrpc bridge stp bonding dm_round_robin dm_multipath scsi_dh loop snd_pcm snd_timer snd soundcore snd_page_alloc serio_raw evdev tpm_tis tpm tpm_bios p smouse pcspkr amd64_edac_mod edac_core button edac_mce_amd shpchp i2c_piix4 container pci_hotplug i2c_core processor ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod sd_mod crc_t10dif mptsas mptscsih mptbase lpfc ehci_hcd scsi_transport_fc tg3 scsi_tgt scsi_transport_sas ohci_hcd libphy scsi_mod usbcore nls_base thermal fan thermal_sys [last unloaded: scsi_wait_scan]
752 Mar 23 14:27:37 sMaster01 kernel: [241450.356084] Pid: 3557, comm: kjournald Not tainted 2.6.32.61vanilla #1 PRIMERGY BX630 S2
753 Mar 23 14:27:37 sMaster01 kernel: [241450.356141] RIP: 0010:[<ffffffff8117e9e9>] [<ffffffff8117e9e9>] bio_check_eod+0x29/0xcd
754 Mar 23 14:27:37 sMaster01 kernel: [241450.356196] RSP: 0018:ffff8804229abba0 EFLAGS: 00010202
755 Mar 23 14:27:37 sMaster01 kernel: [241450.356228] RAX: ffff8804c001fad6 RBX: ffff8802e7235080 RCX: 00011200061e5110
756 Mar 23 14:27:37 sMaster01 kernel: [241450.356279] RDX: 0000000000000008 RSI: 0000000000000008 RDI: ffff8802e7235080
757 Mar 23 14:27:37 sMaster01 kernel: [241450.356331] RBP: ffff8802e7235080 R08: 0000000000000000 R09: ffff880425c54c00
758 Mar 23 14:27:37 sMaster01 kernel: [241450.356383] R10: 0000000000000003 R11: 00000000022e539e R12: ffff8802e7235080
759 Mar 23 14:27:37 sMaster01 kernel: [241450.356434] R13: ffff8802e7235080 R14: ffff880425c54c00 R15: ffff8802e6281850
760 Mar 23 14:27:37 sMaster01 kernel: [241450.356486] FS: 00007faa6a757820(0000) GS:ffff88000fc80000(0000) knlGS:0000000000000000
761 Mar 23 14:27:37 sMaster01 kernel: [241450.356540] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
762 Mar 23 14:27:37 sMaster01 kernel: [241450.356573] CR2: ffff8804c001fade CR3: 00000000cc11f000 CR4: 00000000000006e0
763 Mar 23 14:27:37 sMaster01 kernel: [241450.356628] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
764 Mar 23 14:27:37 sMaster01 kernel: [241450.356681] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
765 Mar 23 14:27:37 sMaster01 kernel: [241450.356733] Process kjournald (pid: 3557, threadinfo ffff8804229aa000, task ffff88041490a300)
766 Mar 23 14:27:37 sMaster01 kernel: [241450.356788] Stack:
767 Mar 23 14:27:37 sMaster01 kernel: [241450.356812] ffff880415382c00 0000000100000285 ffff8804229abfd8 0000000000005186
768 Mar 23 14:27:37 sMaster01 kernel: [241450.356852] <0> 0000000000000000 000000000f1c2776 ffff8804128efa38 ffff8802e7235080
769 Mar 23 14:27:37 sMaster01 kernel: [241450.356913] <0> ffff8802e7235080 ffff8802e7235080 ffff8800cdacae40 ffffffff8117eb5a
770 Mar 23 14:27:37 sMaster01 kernel: [241450.356993] Call Trace:
771 Mar 23 14:27:37 sMaster01 kernel: [241450.357021] [<ffffffff8117eb5a>] ? generic_make_request+0xcd/0x2f9
772 Mar 23 14:27:37 sMaster01 kernel: [241450.357058] [<ffffffff810b6034>] ? mempool_alloc+0x55/0x106
773 Mar 23 14:27:37 sMaster01 kernel: [241450.357091] [<ffffffff8117ee5c>] ? submit_bio+0xd6/0xf2
774 Mar 23 14:27:37 sMaster01 kernel: [241450.357125] [<ffffffff8110d83f>] ? submit_bh+0xf5/0x115
775 Mar 23 14:27:37 sMaster01 kernel: [241450.357158] [<ffffffff8110edc0>] ? sync_dirty_buffer+0x51/0x93
776 Mar 23 14:27:37 sMaster01 kernel: [241450.357196] [<ffffffffa01727c7>] ? journal_commit_transaction+0xaa6/0xe4f [jbd]
777 Mar 23 14:27:37 sMaster01 kernel: [241450.357252] [<ffffffffa0175194>] ? kjournald+0xdf/0x226 [jbd]
778 Mar 23 14:27:37 sMaster01 kernel: [241450.357288] [<ffffffff810651de>] ? autoremove_wake_function+0x0/0x2e
779 Mar 23 14:27:37 sMaster01 kernel: [241450.357324] [<ffffffffa01750b5>] ? kjournald+0x0/0x226 [jbd]
780 Mar 23 14:27:37 sMaster01 kernel: [241450.357357] [<ffffffff81064f11>] ? kthread+0x79/0x81
781 Mar 23 14:27:37 sMaster01 kernel: [241450.357391] [<ffffffff81011baa>] ? child_rip+0xa/0x20
782 Mar 23 14:27:37 sMaster01 kernel: [241450.357425] [<ffffffff81016568>] ? read_tsc+0xa/0x20
783 Mar 23 14:27:37 sMaster01 kernel: [241450.357456] [<ffffffff81064e98>] ? kthread+0x0/0x81
784 Mar 23 14:27:37 sMaster01 kernel: [241450.357487] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
785 Mar 23 14:27:37 sMaster01 kernel: [241450.357517] Code: 5c c3 41 55 49 89 fd 41 54 55 53 48 83 ec 38 65 48 8b 04 25 28 00 00 00 48 89 44 24 28 31 c0 85 f6 0f 84 86 00 00 00 48 8b 47 10 <48> 8b 40 08 48 8b 40 68 48 c1 f8 09 74 74 89 f2 48 8b 0f 48 39
786 Mar 23 14:27:37 sMaster01 kernel: [241450.357738] RIP [<ffffffff8117e9e9>] bio_check_eod+0x29/0xcd
787 Mar 23 14:27:37 sMaster01 kernel: [241450.357772] RSP <ffff8804229abba0>
788 Mar 23 14:27:37 sMaster01 kernel: [241450.357799] CR2: ffff8804c001fade
789 Mar 23 14:27:37 sMaster01 kernel: [241450.358183] ---[ end trace 608fcf1f5a482549 ]---
We had a guest crashing with the same error "unable to handle kernel paging request", but in the function __destroy_inode this time.
Could faulty memory cause this problem on host and guest?
Raph
_______________________________________________ libvirt-users mailing list libvirt-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvirt-users
