Re: [PATCH] proc/meminfo: add MemKernel counter

[Konstantin Khlebnikov <khlebnikov@xxxxxxxxxxxxxx>]

On 22.05.2019 18:01, Vlastimil Babka wrote:
> On 5/22/19 4:40 PM, Konstantin Khlebnikov wrote:
> > Some kinds of kernel allocations are not accounted or not show in meminfo.
> > For example vmalloc allocations are tracked but overall size is not shown
>
> I think Roman's vmalloc patch [1] is on its way?
>
> > for performance reasons. There is no information about network buffers.
>
> xfs buffers can also occupy a lot, from my experience
>
> > In most cases detailed statistics is not required. At first place we need
> > information about overall kernel memory usage regardless of its structure.
> >
> > This patch estimates kernel memory usage by subtracting known sizes of
> > free, anonymous, hugetlb and caches from total memory size: MemKernel =
> > MemTotal - MemFree - Buffers - Cached - SwapCached - AnonPages - Hugetlb.
> >
> > Signed-off-by: Konstantin Khlebnikov <khlebnikov@xxxxxxxxxxxxxx>
>
> I've tried this once in [2]. The name was Unaccounted and one of the objections
> was that people would get worried. Yours is a bit better, perhaps MemKernMisc
> would be even more descriptive? Michal Hocko worried about maintainability, that
> we forget something, but I don't think that's a big issue.

I've started with Misc/Unaccounted too
https://lore.kernel.org/lkml/155792098821.1536.17069603544573830315.stgit@buzz/

But this version simply shows all kernel memory.

> Vlastimil
>
> [1] https://lore.kernel.org/linux-mm/20190514235111.2817276-2-guro@xxxxxx/T/#u
> [2] https://lore.kernel.org/linux-mm/20161020121149.9935-1-vbabka@xxxxxxx/T/#u
>
> > ---
> >   Documentation/filesystems/proc.txt |    5 +++++
> >   fs/proc/meminfo.c                  |   20 +++++++++++++++-----
> >   2 files changed, 20 insertions(+), 5 deletions(-)
> >
> > diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
> > index 66cad5c86171..a0ab7f273ea0 100644
> > --- a/Documentation/filesystems/proc.txt
> > +++ b/Documentation/filesystems/proc.txt
> > @@ -860,6 +860,7 @@ varies by architecture and compile options.  The following is from a
> >   
> >   MemTotal:     16344972 kB
> >   MemFree:      13634064 kB
> > +MemKernel:      862600 kB
> >   MemAvailable: 14836172 kB
> >   Buffers:          3656 kB
> >   Cached:        1195708 kB
> > @@ -908,6 +909,10 @@ MemAvailable: An estimate of how much memory is available for starting new
> >                 page cache to function well, and that not all reclaimable
> >                 slab will be reclaimable, due to items being in use. The
> >                 impact of those factors will vary from system to system.
> > +   MemKernel: The sum of all kinds of kernel memory allocations: Slab,
> > +              Vmalloc, Percpu, KernelStack, PageTables, socket buffers,
> > +              and some other untracked allocations. Does not include
> > +              MemFree, Buffers, Cached, SwapCached, AnonPages, Hugetlb.
> >        Buffers: Relatively temporary storage for raw disk blocks
> >                 shouldn't get tremendously large (20MB or so)
> >         Cached: in-memory cache for files read from the disk (the
> > diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c
> > index 568d90e17c17..b27d56dd619a 100644
> > --- a/fs/proc/meminfo.c
> > +++ b/fs/proc/meminfo.c
> > @@ -39,17 +39,27 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> >   	long available;
> >   	unsigned long pages[NR_LRU_LISTS];
> >   	unsigned long sreclaimable, sunreclaim;
> > +	unsigned long anon_pages, file_pages, swap_cached;
> > +	long kernel_pages;
> >   	int lru;
> >   
> >   	si_meminfo(&i);
> >   	si_swapinfo(&i);
> >   	committed = percpu_counter_read_positive(&vm_committed_as);
> >   
> > -	cached = global_node_page_state(NR_FILE_PAGES) -
> > -			total_swapcache_pages() - i.bufferram;
> > +	anon_pages = global_node_page_state(NR_ANON_MAPPED);
> > +	file_pages = global_node_page_state(NR_FILE_PAGES);
> > +	swap_cached = total_swapcache_pages();
> > +
> > +	cached = file_pages - swap_cached - i.bufferram;
> >   	if (cached < 0)
> >   		cached = 0;
> >   
> > +	kernel_pages = i.totalram - i.freeram - anon_pages - file_pages -
> > +		       hugetlb_total_pages();
> > +	if (kernel_pages < 0)
> > +		kernel_pages = 0;
> > +
> >   	for (lru = LRU_BASE; lru < NR_LRU_LISTS; lru++)
> >   		pages[lru] = global_node_page_state(NR_LRU_BASE + lru);
> >   
> > @@ -60,9 +70,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> >   	show_val_kb(m, "MemTotal:       ", i.totalram);
> >   	show_val_kb(m, "MemFree:        ", i.freeram);
> >   	show_val_kb(m, "MemAvailable:   ", available);
> > +	show_val_kb(m, "MemKernel:      ", kernel_pages);
> >   	show_val_kb(m, "Buffers:        ", i.bufferram);
> >   	show_val_kb(m, "Cached:         ", cached);
> > -	show_val_kb(m, "SwapCached:     ", total_swapcache_pages());
> > +	show_val_kb(m, "SwapCached:     ", swap_cached);
> >   	show_val_kb(m, "Active:         ", pages[LRU_ACTIVE_ANON] +
> >   					   pages[LRU_ACTIVE_FILE]);
> >   	show_val_kb(m, "Inactive:       ", pages[LRU_INACTIVE_ANON] +
> > @@ -92,8 +103,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> >   		    global_node_page_state(NR_FILE_DIRTY));
> >   	show_val_kb(m, "Writeback:      ",
> >   		    global_node_page_state(NR_WRITEBACK));
> > -	show_val_kb(m, "AnonPages:      ",
> > -		    global_node_page_state(NR_ANON_MAPPED));
> > +	show_val_kb(m, "AnonPages:      ", anon_pages);
> >   	show_val_kb(m, "Mapped:         ",
> >   		    global_node_page_state(NR_FILE_MAPPED));
> >   	show_val_kb(m, "Shmem:          ", i.sharedram);