Hi, I've been having some issues with KVM recently where one or more vms will cause page allocation failure messages, usually with the backtrace including networking functions, example follows: [362409.429944] kvm: page allocation failure: order:1, mode:0x20 [362409.429957] Pid: 3453, comm: kvm Not tainted 3.0.0-1-amd64 #1 [362409.429965] Call Trace: [362409.429970] <IRQ> [<ffffffff810b9c90>] ? warn_alloc_failed+0x108/0x11b [362409.429998] [<ffffffff810bcd78>] ? __alloc_pages_nodemask+0x6e6/0x75c [362409.430012] [<ffffffff810ec0c0>] ? kmem_getpages+0x55/0xf0 [362409.430022] [<ffffffff810ec87a>] ? fallback_alloc+0x129/0x1c1 [362409.430035] [<ffffffff8100e28d>] ? paravirt_read_tsc+0x5/0x8 [362409.430045] [<ffffffff810ed10e>] ? kmem_cache_alloc+0x73/0xf0 [362409.430057] [<ffffffff812707a2>] ? sk_prot_alloc+0x2b/0x128 [362409.430067] [<ffffffff81270965>] ? sk_clone+0x14/0x2bd [362409.430077] [<ffffffff812ade7d>] ? inet_csk_clone+0x10/0x91 [362409.430088] [<ffffffff812c1aae>] ? tcp_create_openreq_child+0x21/0x41a [362409.430099] [<ffffffff812bf98a>] ? tcp_v4_syn_recv_sock+0x33/0x208 [362409.430110] [<ffffffff812c2441>] ? tcp_check_req+0x1ff/0x2dd [362409.430122] [<ffffffff812adc06>] ? inet_csk_search_req+0x35/0xa7 [362409.430132] [<ffffffff812bf4f1>] ? tcp_v4_do_rcv+0x206/0x32c [362409.430144] [<ffffffff812c15d4>] ? tcp_v4_rcv+0x419/0x66c [362409.430154] [<ffffffff8100e74a>] ? native_sched_clock+0x28/0x30 [362409.430173] [<ffffffff812a5a0c>] ? ip_local_deliver_finish+0x14b/0x1bb [362409.430186] [<ffffffff8127cc8f>] ? __netif_receive_skb+0x3d7/0x40b [362409.430197] [<ffffffff8127d74b>] ? netif_receive_skb+0x52/0x58 [362409.430220] [<ffffffffa04b5af6>] ? br_nf_pre_routing_finish+0x1d4/0x1e1 [bridge] [362409.430241] [<ffffffffa04b5111>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge] [362409.430260] [<ffffffffa04b60ed>] ? br_nf_pre_routing+0x3be/0x3cb [bridge] [362409.430272] [<ffffffff8129fb69>] ? nf_iterate+0x41/0x77 [362409.430288] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [362409.430305] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [362409.430315] [<ffffffff8129fc12>] ? nf_hook_slow+0x73/0x111 [362409.430330] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [362409.430342] [<ffffffff8103f0a4>] ? try_to_wake_up+0x199/0x199 [362409.430358] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge] [362409.430375] [<ffffffffa04b1389>] ? NF_HOOK.clone.4+0x3c/0x56 [bridge] [362409.430392] [<ffffffffa04b1745>] ? br_handle_frame+0x1af/0x1c6 [bridge] [362409.430408] [<ffffffffa04b1596>] ? br_handle_frame_finish+0x1f3/0x1f3 [bridge] [362409.430420] [<ffffffff8127cb7c>] ? __netif_receive_skb+0x2c4/0x40b [362409.430432] [<ffffffff8127cd3b>] ? process_backlog+0x78/0x157 [362409.430443] [<ffffffff8127dd68>] ? net_rx_action+0xa4/0x1b2 [362409.430454] [<ffffffff81038189>] ? test_tsk_need_resched+0xe/0x17 [362409.430465] [<ffffffff8104bdd4>] ? __do_softirq+0xb9/0x178 [362409.430476] [<ffffffff8133cf1c>] ? call_softirq+0x1c/0x30 [362409.430481] <EOI> [<ffffffff8100a9ef>] ? do_softirq+0x3f/0x84 [362409.430498] [<ffffffff8127e051>] ? netif_rx_ni+0x1e/0x27 [362409.430509] [<ffffffffa06af7a8>] ? tun_get_user+0x390/0x3b8 [tun] [362409.430520] [<ffffffff81060000>] ? bit_waitqueue+0x71/0xa4 [362409.430529] [<ffffffff81023145>] ? _flat_send_IPI_mask+0x6a/0x7c [362409.430541] [<ffffffffa06afbad>] ? tun_get_socket+0x3b/0x3b [tun] [362409.430552] [<ffffffffa06afc0b>] ? tun_chr_aio_write+0x5e/0x79 [tun] [362409.430563] [<ffffffff810fc193>] ? do_sync_readv_writev+0x9a/0xd5 [362409.430574] [<ffffffff810383fc>] ? need_resched+0x1a/0x23 [362409.430585] [<ffffffff813358d7>] ? _cond_resched+0x9/0x20 [362409.430596] [<ffffffff810fb603>] ? copy_from_user+0x18/0x30 [362409.430608] [<ffffffff811653e9>] ? security_file_permission+0x18/0x33 [362409.430618] [<ffffffff810fc3e1>] ? do_readv_writev+0xa4/0x11a [362409.430627] [<ffffffff810fcb94>] ? fput+0x1a/0x1a2 [362409.430636] [<ffffffff810fc5c5>] ? sys_writev+0x45/0x90 [362409.430647] [<ffffffff8133bd12>] ? system_call_fastpath+0x16/0x1b [362409.430654] Mem-Info: [362409.430659] Node 0 DMA per-cpu: [362409.430667] CPU 0: hi: 0, btch: 1 usd: 0 [362409.430673] CPU 1: hi: 0, btch: 1 usd: 0 [362409.430679] CPU 2: hi: 0, btch: 1 usd: 0 [362409.430686] CPU 3: hi: 0, btch: 1 usd: 0 [362409.430691] Node 0 DMA32 per-cpu: [362409.430699] CPU 0: hi: 186, btch: 31 usd: 147 [362409.430705] CPU 1: hi: 186, btch: 31 usd: 82 [362409.430712] CPU 2: hi: 186, btch: 31 usd: 171 [362409.430718] CPU 3: hi: 186, btch: 31 usd: 86 [362409.430724] Node 0 Normal per-cpu: [362409.430730] CPU 0: hi: 186, btch: 31 usd: 157 [362409.430737] CPU 1: hi: 186, btch: 31 usd: 96 [362409.430743] CPU 2: hi: 186, btch: 31 usd: 163 [362409.430749] CPU 3: hi: 186, btch: 31 usd: 173 [362409.430764] active_anon:576670 inactive_anon:110462 isolated_anon:0 [362409.430769] active_file:615766 inactive_file:638287 isolated_file:0 [362409.430774] unevictable:0 dirty:3203 writeback:0 unstable:0 [362409.430778] free:31113 slab_reclaimable:36977 slab_unreclaimable:11009 [362409.430783] mapped:11738 shmem:226 pagetables:9104 bounce:0 [362409.430791] Node 0 DMA free:15912kB min:128kB low:160kB high:192kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [362409.430827] lowmem_reserve[]: 0 3254 8051 8051 [362409.430837] Node 0 DMA32 free:57588kB min:27260kB low:34072kB high:40888kB active_anon:454620kB inactive_anon:92920kB active_file:1279680kB inactive_file:1348708kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3332192kB mlocked:0kB dirty:5928kB writeback:0kB mapped:17404kB shmem:572kB slab_reclaimable:71792kB slab_unreclaimable:5488kB kernel_stack:296kB pagetables:2420kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [362409.430877] lowmem_reserve[]: 0 0 4797 4797 [362409.430886] Node 0 Normal free:51448kB min:40192kB low:50240kB high:60288kB active_anon:1852060kB inactive_anon:348928kB active_file:1183384kB inactive_file:1204152kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4912640kB mlocked:0kB dirty:6884kB writeback:0kB mapped:29548kB shmem:332kB slab_reclaimable:76116kB slab_unreclaimable:38548kB kernel_stack:3184kB pagetables:33996kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9 all_unreclaimable? no [362409.430926] lowmem_reserve[]: 0 0 0 0 [362409.430935] Node 0 DMA: 0*4kB 1*8kB 0*16kB 1*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15912kB [362409.430959] Node 0 DMA32: 13011*4kB 198*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 57836kB [362409.430983] Node 0 Normal: 11740*4kB 49*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 51448kB [362409.431007] 1269832 total pagecache pages [362409.431012] 15734 pages in swap cache [362409.431018] Swap cache stats: add 129275, delete 113541, find 78987/83931 [362409.431025] Free swap = 7546528kB [362409.431030] Total swap = 7811068kB [362409.433896] 2097136 pages RAM [362409.433896] 47949 pages reserved [362409.433896] 681769 pages shared [362409.433896] 1367619 pages non-shared The server has 8G of ram, and usually never uses more than about 4G (sitting at 3.4G right now). -- Thomas Fjellstrom thomas@xxxxxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html