OOM in v4.8

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

There is a chromeswap test case:
https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/client/site_tests/platform_CompressedSwapPerf

We have done small changes and ported it to our LKP environment:
https://github.com/aaronlu/chromeswap

The test starts nr_procs processes and let them each allocate some
memory equally with realloc, so anonymous pages are used. When the
pre-specified swap_target is reached, the allocation will stop. The
total allocation size is: MemFree + swap_target * SwapTotal.
After allocation, a random process is selected to touch its memory to
trigger swap in/out.

For this test, nr_procs is 50 and swap_target is 50%.
The test box has 8G memory where 4G is used as a pmem block device and
created as the swap partition.

There is OOM occured for this test recently so I did more tests:
on v4.6, 10 tests all pass;
on v4.7, 2 tests OOMed out of 10 tests;
on v4.8, 6 tests OOMed out of 10 tests;
on 101105b1717f, which is yersterday's Linus' master branch head,
1 test OOMed out of 10 tests.

SO things are much better than v4.8 now.

When OOM occurred, there is still enough swap space though:

kern  :warn  : [   38.708419] proc-vmstat invoked oom-killer: gfp_mask=0x27000c0(GFP_KERNEL_ACCOUNT|__GFP_NOTRACK), order=2, oom_score_adj=0
kern  :info  : [   38.720644] proc-vmstat cpuset=/ mems_allowed=0
kern  :warn  : [   38.726404] CPU: 5 PID: 500 Comm: proc-vmstat Not tainted 4.8.0 #1
kern  :warn  : [   38.733731] Hardware name: Dell Inc. OptiPlex 9020/0DNKMN, BIOS A05 12/05/2013
kern  :warn  : [   38.742114]  0000000000000000 ffff8800c106fb60 ffffffff8144f659 ffff8800c106fcf0
kern  :warn  : [   38.750709]  ffff88021c2325c0 ffff8800c106fbc8 ffffffff81209a0c 01018800c106fb70
kern  :warn  : [   38.759267]  ffffffff81ee14a0 0000000000000015 ffffffff81e43340 0000000000000206
kern  :warn  : [   38.767794] Call Trace:
kern  :warn  : [   38.771323]  [<ffffffff8144f659>] dump_stack+0x63/0x8a
kern  :warn  : [   38.777520]  [<ffffffff81209a0c>] dump_header+0x5c/0x1ff
kern  :warn  : [   38.783874]  [<ffffffff81181e3c>] oom_kill_process+0x22c/0x410
kern  :warn  : [   38.790731]  [<ffffffff810886a5>] ? has_capability_noaudit+0x25/0x40
kern  :warn  : [   38.798140]  [<ffffffff8118248a>] out_of_memory+0x41a/0x430
kern  :warn  : [   38.804722]  [<ffffffff811877db>] __alloc_pages_slowpath+0xa7b/0xaa0
kern  :warn  : [   38.812034]  [<ffffffff81187aab>] __alloc_pages_nodemask+0x2ab/0x2f0
kern  :warn  : [   38.819344]  [<ffffffff8107bf2e>] copy_process+0x11e/0x1990
kern  :warn  : [   38.826558]  [<ffffffff811e7696>] ? kmem_cache_alloc+0x1a6/0x1c0
kern  :warn  : [   38.833483]  [<ffffffff813e4427>] ? selinux_file_alloc_security+0x37/0x60
kern  :warn  : [   38.841172]  [<ffffffff813e4427>] ? selinux_file_alloc_security+0x37/0x60
kern  :warn  : [   38.848858]  [<ffffffff8107d96a>] _do_fork+0xca/0x3f0
kern  :warn  : [   38.854793]  [<ffffffff8122d367>] ? __fd_install+0x37/0x100
kern  :warn  : [   38.861231]  [<ffffffff8107dd39>] SyS_clone+0x19/0x20
kern  :warn  : [   38.867122]  [<ffffffff81003bb7>] do_syscall_64+0x67/0x160
kern  :warn  : [   38.873459]  [<ffffffff819341e1>] entry_SYSCALL64_slow_path+0x25/0x25
kern  :warn  : [   38.880744] Mem-Info:
kern  :warn  : [   38.883875] active_anon:622526 inactive_anon:154230 isolated_anon:0
                               active_file:0 inactive_file:1 isolated_file:0
                               unevictable:94198 dirty:0 writeback:0 unstable:3
                               slab_reclaimable:59989 slab_unreclaimable:6489
                               mapped:6022 shmem:257 pagetables:3956 bounce:0
                               free:17325 free_pcp:357 free_cma:897
kern  :warn  : [   38.920992] Node 0 active_anon:2477952kB inactive_anon:619360kB active_file:0kB inactive_file:4kB unevictable:376792kB isolated(anon):0kB isolated(file):0kB mapped:24088kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 12288kB anon_thp: 1028kB writeback_tmp:0kB unstable:12kB pages_scanned:0 all_unreclaimable? no
kern  :warn  : [   38.952034] Node 0 DMA free:2008kB min:280kB low:348kB high:416kB active_anon:1112kB inactive_anon:28kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15984kB managed:15900kB mlocked:0kB slab_reclaimable:12704kB slab_unreclaimable:48kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
kern  :warn  : [   38.984654] lowmem_reserve[]: 0 3430 3524 3524 3524
kern  :warn  : [   38.990371] Node 0 DMA32 free:83292kB min:61984kB low:77480kB high:92976kB active_anon:2395900kB inactive_anon:454596kB active_file:0kB inactive_file:4kB unevictable:314016kB writepending:0kB present:3578492kB managed:3512924kB mlocked:92kB slab_reclaimable:218640kB slab_unreclaimable:6036kB kernel_stack:1744kB pagetables:14040kB bounce:0kB free_pcp:2160kB local_pcp:36kB free_cma:0kB
kern  :warn  : [   39.027044] lowmem_reserve[]: 0 0 94 94 94
kern  :warn  : [   39.031921] Node 0 Normal free:5448kB min:5316kB low:6644kB high:7972kB active_anon:61364kB inactive_anon:162752kB active_file:0kB inactive_file:0kB unevictable:62776kB writepending:0kB present:505856kB managed:420724kB mlocked:2124kB slab_reclaimable:17396kB slab_unreclaimable:19876kB kernel_stack:2992kB pagetables:1784kB bounce:0kB free_pcp:60kB local_pcp:0kB free_cma:4004kB
kern  :warn  : [   39.067344] lowmem_reserve[]: 0 0 0 0 0
kern  :warn  : [   39.072034] Node 0 DMA: 47*4kB (E) 9*8kB (E) 3*16kB (H) 1*32kB (H) 1*64kB (H) 1*128kB (H) 0*256kB 1*512kB (H) 1*1024kB (H) 0*2048kB 0*4096kB = 2068kB
kern  :warn  : [   39.087289] Node 0 DMA32: 9514*4kB (UME) 3085*8kB (ME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 62736kB
kern  :warn  : [   39.101342] Node 0 Normal: 203*4kB (UEC) 400*8kB (UEC) 107*16kB (HC) 5*32kB (C) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 5884kB
kern  :info  : [   39.116367] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
kern  :info  : [   39.126827] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
kern  :warn  : [   39.136481] 94403 total pagecache pages
kern  :warn  : [   39.141514] 10 pages in swap cache
kern  :warn  : [   39.145995] Swap cache stats: add 6689320, delete 6689310, find 6080/3054405
kern  :warn  : [   39.154135] Free swap  = 1845896kB
kern  :warn  : [   39.158638] Total swap = 4194300kB
kern  :warn  : [   39.163152] 1025083 pages RAM
kern  :warn  : [   39.167218] 0 pages HighMem/MovableOnly
kern  :warn  : [   39.172167] 37696 pages reserved
kern  :warn  : [   39.176504] 51200 pages cma reserved
kern  :warn  : [   39.181223] 0 pages hwpoisoned

I wonder if this OOM could/should be avoided?

Full dmesg for v4.7, v4.8 and 101105b1717f are attached, please let me
know if you need more information.

Thanks,
Aaron

Attachment: v4.7.xz
Description: application/xz

Attachment: v4.8.xz
Description: application/xz

Attachment: 101105b1717f.xz
Description: application/xz


[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [ECOS]     [Asterisk Internet PBX]     [Linux API]