Hi,
we have a couple of nodes based on CentOS 6 and Xen4CentOS. Unfortunately
some of these nodes keep crashing frequenly.
We use the latest versions:
# uname -r
3.10.43-11.el6.centos.alt.x86_64
# xm info
host : vserver20
release : 3.10.43-11.el6.centos.alt.x86_64
version : #1 SMP Mon Jun 16 14:22:02 UTC 2014
machine : x86_64
nr_cpus : 24
nr_nodes : 2
cores_per_socket : 6
threads_per_core : 2
cpu_mhz : 2400
hw_caps : bfebfbff:2c100800:00000000:00003f40:009ee3fd:00000000:00000001:00000000
virt_caps : hvm hvm_directio
total_memory : 65527
free_memory : 22692
free_cpus : 0
xen_major : 4
xen_minor : 2
xen_extra : .4-33.el6
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline : dom0_mem=2560M,max:3072M loglvl=all
guest_loglvl=all
cc_compiler : gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
cc_compile_by : mockbuild
cc_compile_domain : centos.org
cc_compile_date : Mon Jun 16 17:22:14 UTC 2014
xend_config_format : 4
Our configuration looks as follows:
Grub:
title CentOS (3.10.43-11.el6.centos.alt.x86_64)
root (hd0,1)
kernel /xen.gz dom0_mem=2560M,max:3072M loglvl=all guest_loglvl=all
module /vmlinuz-3.10.43-11.el6.centos.alt.x86_64 ro root=/dev/sda1 KEYBOARDTYPE=pc KEYTABLE=de-latin1-nodeadkeys crashkernel=auto
module /initramfs-3.10.43-11.el6.centos.alt.x86_64.img
/etc/xen/xend-config.sxp
(xend-unix-server yes)
(xend-relocation-server no)
(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 1024)
(enable-dom0-ballooning no)
(total_available_memory 0)
(dom0-cpus 0)
(vncpasswd '')
I've attached the logfile information regarding the latest crash as
crash.log?
Does anybody has an idea how to solve these issues?
Kind Regards
Daniel BradlerJul 29 18:50:04 vserver20 kernel: BUG: unable to handle kernel paging request at 000000660000008c
Jul 29 18:50:04 vserver20 kernel: IP: [<ffffffff81151999>] isolate_migratepages_range+0x459/0x980
Jul 29 18:50:04 vserver20 kernel: PGD 2c3d4067 PUD 0
Jul 29 18:50:04 vserver20 kernel: Oops: 0000 [#1] SMP
Jul 29 18:50:04 vserver20 kernel: Modules linked in: bridge stp llc xen_pciback xen_gntalloc xt_REDIRECT xt_owner nf_nat_ftp nf_conntrack_ftp xt_state xt_length xt_hl xt_tcpmss xt_TCPMSS xt_multiport xt_limit xt_LOG xt_DSCP xt_dscp ipt_REJECT iptable_filter iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables ip6table_filter ip6_tables ipv6 xen_acpi_processor blktap xen_netback xen_blkback xen_gntdev xen_evtchn xenfs xen_privcmd gpio_ich iTCO_wdt iTCO_vendor_support coretemp hwmon freq_table mperf intel_powerclamp crc32c_intel microcode serio_raw pcspkr i2c_i801 joydev lpc_ich e1000e ptp pps_core ioatdma dca i7core_edac edac_core sg ext3 jbd mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix aacraid mgag200 ttm drm_kms_helper dm_mirror dm_region_hash dm_log dm_mod
Jul 29 18:50:04 vserver20 kernel: CPU: 21 PID: 17034 Comm: solusvmc-node Not tainted 3.10.43-11.el6.centos.alt.x86_64 #1
Jul 29 18:50:04 vserver20 kernel: Hardware name: Supermicro X8DTL/X8DTL, BIOS 2.1b 11/16/2012
Jul 29 18:50:04 vserver20 kernel: task: ffff880003e31540 ti: ffff88000011e000 task.ti: ffff88000011e000
Jul 29 18:50:04 vserver20 kernel: RIP: e030:[<ffffffff81151999>] [<ffffffff81151999>] isolate_migratepages_range+0x459/0x980
Jul 29 18:50:04 vserver20 kernel: RSP: e02b:ffff88000011f990 EFLAGS: 00010206
Jul 29 18:50:04 vserver20 kernel: RAX: 0000006600000014 RBX: 0000000000001db0 RCX: 000000000000000e
Jul 29 18:50:04 vserver20 kernel: RDX: 0000000000000002 RSI: 0000000000000003 RDI: 000000000000003b
Jul 29 18:50:04 vserver20 kernel: RBP: ffff88000011fa40 R08: ffffea0000000000 R09: 0000000000001e00
Jul 29 18:50:04 vserver20 kernel: R10: ffff8800a003eb40 R11: ffffea0000062000 R12: ffffea0000067e80
Jul 29 18:50:04 vserver20 kernel: R13: 00000000000001b1 R14: 0000000000000000 R15: ffff8800a003e6c0
Jul 29 18:50:04 vserver20 kernel: FS: 00007f38bffdf700(0000) GS:ffff88009f2a0000(0000) knlGS:0000000000000000
Jul 29 18:50:04 vserver20 kernel: CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Jul 29 18:50:04 vserver20 kernel: CR2: 000000660000008c CR3: 0000000020e7b000 CR4: 0000000000002660
Jul 29 18:50:04 vserver20 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul 29 18:50:04 vserver20 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul 29 18:50:04 vserver20 kernel: Stack:
Jul 29 18:50:04 vserver20 kernel: 000000000000000e ffff8800a003eb40 ffffea0000000000 ffffea0000062000
Jul 29 18:50:04 vserver20 kernel: 0000000000001e00 0000000000000015 00ff88009f2b00a0 ffff880003e31540
Jul 29 18:50:04 vserver20 kernel: 0000000000100000 0000000000000000 ffff88000011fad0 0000000000000000
Jul 29 18:50:04 vserver20 kernel: Call Trace:
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81152ac0>] ? isolate_freepages+0x270/0x270
Jul 29 18:50:04 vserver20 kernel: [<ffffffff8115218e>] compact_zone+0x2ce/0x450
Jul 29 18:50:04 vserver20 kernel: [<ffffffff811526f2>] compact_zone_order+0xa2/0xf0
Jul 29 18:50:04 vserver20 kernel: [<ffffffff815f7ec7>] ? _raw_spin_unlock_irqrestore+0x17/0x20
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81152811>] try_to_compact_pages+0xd1/0x110
Jul 29 18:50:04 vserver20 kernel: [<ffffffff811380d0>] __alloc_pages_direct_compact+0x90/0x200
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81138562>] __alloc_pages_slowpath+0x322/0x7c0
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81138d0e>] __alloc_pages_nodemask+0x30e/0x330
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81005269>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81058518>] dup_task_struct+0x68/0x250
Jul 29 18:50:04 vserver20 kernel: [<ffffffff8105997d>] copy_process+0xfd/0xe50
Jul 29 18:50:04 vserver20 kernel: [<ffffffff811b7ee9>] ? mntput_no_expire+0x49/0x150
Jul 29 18:50:04 vserver20 kernel: [<ffffffff8105aa6a>] do_fork+0x4a/0x200
Jul 29 18:50:04 vserver20 kernel: [<ffffffff8105ac36>] SyS_clone+0x16/0x20
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81601239>] stub_clone+0x69/0x90
Jul 29 18:50:04 vserver20 kernel: [<ffffffff81600ed9>] ? system_call_fastpath+0x16/0x1b
Jul 29 18:50:04 vserver20 kernel: Code: ff 49 8b 04 24 66 85 c0 0f 88 b3 04 00 00 4c 89 e0 8b 40 1c 83 f8 01 0f 85 59 fd ff ff 49 8b 44 24 08 48 85 c0 0f 84 4b fd ff ff <48> 8b 40 78 a9 00 00 00 20 0f 84 3c fd ff ff 45 84 f6 0f 84 33
Jul 29 18:50:04 vserver20 kernel: RIP [<ffffffff81151999>] isolate_migratepages_range+0x459/0x980
Jul 29 18:50:04 vserver20 kernel: RSP <ffff88000011f990>
Jul 29 18:50:04 vserver20 kernel: CR2: 000000660000008c
Jul 29 18:50:04 vserver20 kernel: ---[ end trace d4ce8915a3622ed9 ]---
Jul 29 18:50:30 vserver20 kernel: BUG: unable to handle kernel paging request at 000000220000008c
Jul 29 18:50:30 vserver20 kernel: IP: [<ffffffff81151999>] isolate_migratepages_range+0x459/0x980
Jul 29 18:50:30 vserver20 kernel: PGD d88b067 PUD 0
Jul 29 18:50:30 vserver20 kernel: Oops: 0000 [#2] SMP
Jul 29 18:50:30 vserver20 kernel: Modules linked in: bridge stp llc xen_pciback xen_gntalloc xt_REDIRECT xt_owner nf_nat_ftp nf_conntrack_ftp xt_state xt_length xt_hl xt_tcpmss xt_TCPMSS xt_multiport xt_limit xt_LOG xt_DSCP xt_dscp ipt_REJECT iptable_filter iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables ip6table_filter ip6_tables ipv6 xen_acpi_processor blktap xen_netback xen_blkback xen_gntdev xen_evtchn xenfs xen_privcmd gpio_ich iTCO_wdt iTCO_vendor_support coretemp hwmon freq_table mperf intel_powerclamp crc32c_intel microcode serio_raw pcspkr i2c_i801 joydev lpc_ich e1000e ptp pps_core ioatdma dca i7core_edac edac_core sg ext3 jbd mbcache sd_mod crc_t10dif pata_acpi ata_generic ata_piix aacraid mgag200 ttm drm_kms_helper dm_mirror dm_region_hash dm_log dm_mod
Jul 29 18:50:30 vserver20 kernel: CPU: 15 PID: 23546 Comm: sh Tainted: G D 3.10.43-11.el6.centos.alt.x86_64 #1
Jul 29 18:50:30 vserver20 kernel: Hardware name: Supermicro X8DTL/X8DTL, BIOS 2.1b 11/16/2012
Jul 29 18:50:30 vserver20 kernel: task: ffff88000f08c040 ti: ffff880001dc2000 task.ti: ffff880001dc2000
Jul 29 18:50:30 vserver20 kernel: RIP: e030:[<ffffffff81151999>] [<ffffffff81151999>] isolate_migratepages_range+0x459/0x980
Jul 29 18:50:30 vserver20 kernel: RSP: e02b:ffff880001dc3990 EFLAGS: 00010006
Jul 29 18:50:30 vserver20 kernel: RAX: 0000002200000014 RBX: 0000000000001da8 RCX: 000000000000000e
Jul 29 18:50:30 vserver20 kernel: RDX: 0000000000000002 RSI: 0000000000000003 RDI: 000000000000003b
Jul 29 18:50:30 vserver20 kernel: RBP: ffff880001dc3a40 R08: ffffea0000000000 R09: 0000000000001e00
Jul 29 18:50:30 vserver20 kernel: R10: ffff8800a003eb40 R11: ffffea0000062000 R12: ffffea0000067cc0
Jul 29 18:50:30 vserver20 kernel: R13: 00000000000001a9 R14: 0000000000000001 R15: ffff8800a003e6c0
Jul 29 18:50:30 vserver20 kernel: FS: 00007f38360f8700(0000) GS:ffff88009f1e0000(0000) knlGS:0000000000000000
Jul 29 18:50:30 vserver20 kernel: CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Jul 29 18:50:30 vserver20 kernel: CR2: 000000220000008c CR3: 0000000036cce000 CR4: 0000000000002660
Jul 29 18:50:30 vserver20 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul 29 18:50:30 vserver20 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul 29 18:50:30 vserver20 kernel: Stack:
Jul 29 18:50:30 vserver20 kernel: 000000000000000e ffff8800a003eb40 ffffea0000000000 ffffea0000062000
Jul 29 18:50:30 vserver20 kernel: 0000000000001e00 ffff880096f13208 00ff88009f1f00a0 ffff88000f08c040
Jul 29 18:50:30 vserver20 kernel: 0000000400100000 0000000000000001 ffff880001dc3ad0 0000000000000000
Jul 29 18:50:30 vserver20 kernel: Call Trace:
Jul 29 18:50:30 vserver20 kernel: [<ffffffff8115218e>] compact_zone+0x2ce/0x450
Jul 29 18:50:30 vserver20 kernel: [<ffffffff811526f2>] compact_zone_order+0xa2/0xf0
Jul 29 18:50:30 vserver20 kernel: [<ffffffff81152811>] try_to_compact_pages+0xd1/0x110
Jul 29 18:50:30 vserver20 kernel: [<ffffffff811380d0>] __alloc_pages_direct_compact+0x90/0x200
Jul 29 18:50:30 vserver20 kernel: [<ffffffff81138562>] __alloc_pages_slowpath+0x322/0x7c0
Jul 29 18:50:30 vserver20 kernel: [<ffffffff81138d0e>] __alloc_pages_nodemask+0x30e/0x330
Jul 29 18:50:30 vserver20 kernel: [<ffffffff81058518>] dup_task_struct+0x68/0x250
Jul 29 18:50:30 vserver20 kernel: [<ffffffff8105997d>] copy_process+0xfd/0xe50
Jul 29 18:50:30 vserver20 kernel: [<ffffffff811a4f2b>] ? path_get+0x2b/0x40
Jul 29 18:50:30 vserver20 kernel: [<ffffffff811b5e3d>] ? __alloc_fd+0xcd/0x140
Jul 29 18:50:30 vserver20 kernel: [<ffffffff8105aa6a>] do_fork+0x4a/0x200
Jul 29 18:50:30 vserver20 kernel: [<ffffffff8105ac36>] SyS_clone+0x16/0x20
Jul 29 18:50:30 vserver20 kernel: [<ffffffff81601239>] stub_clone+0x69/0x90
Jul 29 18:50:30 vserver20 kernel: [<ffffffff81600ed9>] ? system_call_fastpath+0x16/0x1b
Jul 29 18:50:30 vserver20 kernel: Code: ff 49 8b 04 24 66 85 c0 0f 88 b3 04 00 00 4c 89 e0 8b 40 1c 83 f8 01 0f 85 59 fd ff ff 49 8b 44 24 08 48 85 c0 0f 84 4b fd ff ff <48> 8b 40 78 a9 00 00 00 20 0f 84 3c fd ff ff 45 84 f6 0f 84 33
Jul 29 18:50:30 vserver20 kernel: RIP [<ffffffff81151999>] isolate_migratepages_range+0x459/0x980
Jul 29 18:50:30 vserver20 kernel: RSP <ffff880001dc3990>
Jul 29 18:50:30 vserver20 kernel: CR2: 000000220000008c
Jul 29 18:50:30 vserver20 kernel: ---[ end trace d4ce8915a3622eda ]---
_______________________________________________
CentOS-virt mailing list
CentOS-virt@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos-virt
