On 3/10/19 8:13 AM, Michal Hocko wrote:
On Fri 08-03-19 21:02:41, Martynas Pumputis wrote:
On 3/8/19 12:20 PM, Michal Hocko wrote:
On Fri 08-03-19 12:14:16, Martynas Pumputis wrote:
On 3/8/19 9:44 AM, Michal Hocko wrote:
On Fri 08-03-19 09:08:57, Martynas Pumputis wrote:
It has been observed that sometimes memory allocation for BPF maps
fails when there is no obvious memory pressure in a system.
E.g. the map (BPF_MAP_TYPE_LRU_HASH, key=38, value=56, max_elems=524288)
could not be created due to due to vmalloc unable to allocate 75497472B,
when the system's memory consumption (in MB) was the following:
Total: 3942 Used: 837 (21.24%) Free: 138 Buffers: 239 Cached: 2727
Hmm 75MB is quite large and much larger than the slab/page allocator
cann provide so this is not really a fragmentation issue. Vmalloc does
respect noretry but considering that there shouldn't be a large memory
pressure I wonder how NORETRY managed to fail the allocation. Do you
happen to have the allocation failure report?
I got /proc/{meminfo,vmstat,vmallocinfo} just after the allocation has
failed:
https://gist.github.com/brb/62092c1d83daa6527271b88f0352e32d
dmesg with the allocation failure report would be more helpful
https://gist.github.com/brb/2d7ac323d2e14cb7a38bacba301fe3af
Thanks!
tc: vmalloc: allocation failure, allocated 15609856 of 62918656 bytes, mode:0x6090c0(GFP_KERNEL|__GFP_NORETRY|__GFP_ZERO), nodemask=(null),cpuset=b389e318420d891300ad9658f8e056b59972fda9547dd566245a922c34bb9e42,mems_allowed=0
[...]
Node 0 DMA free:15728kB min:268kB low:332kB high:396kB active_anon:0kB inactive_anon:0kB active_file:44kB inactive_file:12kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
lowmem_reserve[]: 0 3419 3866 3866 3866
Node 0 DMA32 free:105004kB min:104588kB low:119468kB high:134348kB active_anon:526128kB inactive_anon:612kB active_file:862524kB inactive_file:1552884kB unevictable:0kB writepending:0kB present:3653568kB managed:3563596kB mlocked:0kB kernel_stack:7592kB pagetables:6636kB bounce:0kB free_pcp:916kB local_pcp:736kB free_cma:0kB
lowmem_reserve[]: 0 0 446 446 446
Node 0 Normal free:22844kB min:24160kB low:26104kB high:28048kB active_anon:92340kB inactive_anon:228kB active_file:160072kB inactive_file:82480kB unevictable:0kB writepending:0kB present:524288kB managed:457544kB mlocked:0kB kernel_stack:2224kB pagetables:3776kB bounce:0kB free_pcp:996kB local_pcp:672kB free_cma:0kB
lowmem_reserve[]: 0 0 0 0 0
Except for a srtange cpuset value (which should be checked separately),
the allocation is restricted to node 0 which is pretty much out of
memory (below min watermark - lowmem_reserve). There is still a lot of
page cache to reclaim so a further reclaim is quite likely to make a
further progress. There is still 45MB to go and at least page cache is
1.5G so there is some buffer to allocate from.
That being said __GFP_NORETRY caused a pre-mature failure indeed. Using
kvmalloc(GFP_KERNEL|__GFP_RETRY_MAYFAIL) would likely help here unless
the pagecache is really hard to reclaim. Please note that this will also
imply that requests which can be satisfied from the slab allocator will
retry harder as well. Not sure this is desirable for these requests
though but your original patch does the same so if you wanted to have
__GFP_RETRY_MAYFAIL behavior only for the vmalloc path then you would
need to have an opencoded version which adds the flag just to the
vmalloc fallback path.
Thanks a lot for the analysis. I've re-submitted the patch.
