Am 06.09.19 um 20:52 schrieb Yang Shi: > On Fri, Sep 6, 2019 at 3:08 AM Stefan Priebe - Profihost AG > <s.priebe@xxxxxxxxxxxx> wrote: >> >> These are the biggest differences in meminfo before and after cached >> starts to drop. I didn't expect cached end up in MemFree. >> >> Before: >> MemTotal: 16423116 kB >> MemFree: 374572 kB > > Here MemFree is only ~300MB? It is quite low comparing with the amount > of total memory. It may trigger water mark to launch kswapd. mhm yes that might be possible but i don't see kswapd running in the process list at least not using cpu. Also does it really free all cache? i thought it would only free up to vm.min_free_kbytes but this is 160MB on this machine. >> MemAvailable: 5633816 kB >> Cached: 5550972 kB >> Inactive: 4696580 kB >> Inactive(file): 3624776 kB >> >> >> After: >> MemTotal: 16423116 kB >> MemFree: 3477168 kB > > Here MemFree is ~3GB and file cache was shrunk from ~5G down to ~2G. yes - bu i thought if file cache gets shrunk another process has requested tis memory and would use it. So it does not end up in free. I'm sure this all is explainable - but i really would like to know how. Greets, Stefan >> MemAvailable: 6066916 kB >> Cached: 2724504 kB >> Inactive: 1854740 kB >> Inactive(file): 950680 kB >> >> Any explanation? >> >> Greets, >> Stefan >> Am 05.09.19 um 13:56 schrieb Stefan Priebe - Profihost AG: >>> >>> Am 05.09.19 um 13:40 schrieb Michal Hocko: >>>> On Thu 05-09-19 13:27:10, Stefan Priebe - Profihost AG wrote: >>>>> Hello all, >>>>> >>>>> i hope you can help me again to understand the current MemAvailable >>>>> value in the linux kernel. I'm running a 4.19.52 kernel + psi patches in >>>>> this case. >>>>> >>>>> I'm seeing the following behaviour i don't understand and ask for help. >>>>> >>>>> While MemAvailable shows 5G the kernel starts to drop cache from 4G down >>>>> to 1G while the apache spawns some PHP processes. After that the PSI >>>>> mem.some value rises and the kernel tries to reclaim memory but >>>>> MemAvailable stays at 5G. >>>>> >>>>> Any ideas? >>>> >>>> Can you collect /proc/vmstat (every second or so) and post it while this >>>> is the case please? >>> >>> Yes sure. >>> >>> But i don't know which event you mean exactly. Current situation is PSI >>> / memory pressure is > 20 but: >>> >>> This is the current status where MemAvailable show 5G but Cached is >>> already dropped to 1G coming from 4G: >>> >>> >>> meminfo: >>> MemTotal: 16423116 kB >>> MemFree: 5280736 kB >>> MemAvailable: 5332752 kB >>> Buffers: 2572 kB >>> Cached: 1225112 kB >>> SwapCached: 0 kB >>> Active: 8934976 kB >>> Inactive: 1026900 kB >>> Active(anon): 8740396 kB >>> Inactive(anon): 873448 kB >>> Active(file): 194580 kB >>> Inactive(file): 153452 kB >>> Unevictable: 19900 kB >>> Mlocked: 19900 kB >>> SwapTotal: 0 kB >>> SwapFree: 0 kB >>> Dirty: 1980 kB >>> Writeback: 0 kB >>> AnonPages: 8423480 kB >>> Mapped: 978212 kB >>> Shmem: 875680 kB >>> Slab: 839868 kB >>> SReclaimable: 383396 kB >>> SUnreclaim: 456472 kB >>> KernelStack: 22576 kB >>> PageTables: 49824 kB >>> NFS_Unstable: 0 kB >>> Bounce: 0 kB >>> WritebackTmp: 0 kB >>> CommitLimit: 8211556 kB >>> Committed_AS: 32060624 kB >>> VmallocTotal: 34359738367 kB >>> VmallocUsed: 0 kB >>> VmallocChunk: 0 kB >>> Percpu: 118048 kB >>> HardwareCorrupted: 0 kB >>> AnonHugePages: 6406144 kB >>> ShmemHugePages: 0 kB >>> ShmemPmdMapped: 0 kB >>> HugePages_Total: 0 >>> HugePages_Free: 0 >>> HugePages_Rsvd: 0 >>> HugePages_Surp: 0 >>> Hugepagesize: 2048 kB >>> Hugetlb: 0 kB >>> DirectMap4k: 2580336 kB >>> DirectMap2M: 14196736 kB >>> DirectMap1G: 2097152 kB >>> >>> >>> vmstat shows: >>> nr_free_pages 1320053 >>> nr_zone_inactive_anon 218362 >>> nr_zone_active_anon 2185108 >>> nr_zone_inactive_file 38363 >>> nr_zone_active_file 48645 >>> nr_zone_unevictable 4975 >>> nr_zone_write_pending 495 >>> nr_mlock 4975 >>> nr_page_table_pages 12553 >>> nr_kernel_stack 22576 >>> nr_bounce 0 >>> nr_zspages 0 >>> nr_free_cma 0 >>> numa_hit 13916119899 >>> numa_miss 0 >>> numa_foreign 0 >>> numa_interleave 15629 >>> numa_local 13916119899 >>> numa_other 0 >>> nr_inactive_anon 218362 >>> nr_active_anon 2185164 >>> nr_inactive_file 38363 >>> nr_active_file 48645 >>> nr_unevictable 4975 >>> nr_slab_reclaimable 95849 >>> nr_slab_unreclaimable 114118 >>> nr_isolated_anon 0 >>> nr_isolated_file 0 >>> workingset_refault 71365357 >>> workingset_activate 20281670 >>> workingset_restore 8995665 >>> workingset_nodereclaim 326085 >>> nr_anon_pages 2105903 >>> nr_mapped 244553 >>> nr_file_pages 306921 >>> nr_dirty 495 >>> nr_writeback 0 >>> nr_writeback_temp 0 >>> nr_shmem 218920 >>> nr_shmem_hugepages 0 >>> nr_shmem_pmdmapped 0 >>> nr_anon_transparent_hugepages 3128 >>> nr_unstable 0 >>> nr_vmscan_write 0 >>> nr_vmscan_immediate_reclaim 1833104 >>> nr_dirtied 386544087 >>> nr_written 259220036 >>> nr_dirty_threshold 265636 >>> nr_dirty_background_threshold 132656 >>> pgpgin 1817628997 >>> pgpgout 3730818029 >>> pswpin 0 >>> pswpout 0 >>> pgalloc_dma 0 >>> pgalloc_dma32 5790777997 >>> pgalloc_normal 20003662520 >>> pgalloc_movable 0 >>> allocstall_dma 0 >>> allocstall_dma32 0 >>> allocstall_normal 39 >>> allocstall_movable 1980089 >>> pgskip_dma 0 >>> pgskip_dma32 0 >>> pgskip_normal 0 >>> pgskip_movable 0 >>> pgfree 26637215947 >>> pgactivate 316722654 >>> pgdeactivate 261039211 >>> pglazyfree 0 >>> pgfault 17719356599 >>> pgmajfault 30985544 >>> pglazyfreed 0 >>> pgrefill 286826568 >>> pgsteal_kswapd 36740923 >>> pgsteal_direct 349291470 >>> pgscan_kswapd 36878966 >>> pgscan_direct 395327492 >>> pgscan_direct_throttle 0 >>> zone_reclaim_failed 0 >>> pginodesteal 49817087 >>> slabs_scanned 597956834 >>> kswapd_inodesteal 1412447 >>> kswapd_low_wmark_hit_quickly 39 >>> kswapd_high_wmark_hit_quickly 319 >>> pageoutrun 3585 >>> pgrotated 2873743 >>> drop_pagecache 0 >>> drop_slab 0 >>> oom_kill 0 >>> pgmigrate_success 839062285 >>> pgmigrate_fail 507313 >>> compact_migrate_scanned 9619077010 >>> compact_free_scanned 67985619651 >>> compact_isolated 1684537704 >>> compact_stall 205761 >>> compact_fail 182420 >>> compact_success 23341 >>> compact_daemon_wake 2 >>> compact_daemon_migrate_scanned 811 >>> compact_daemon_free_scanned 490241 >>> htlb_buddy_alloc_success 0 >>> htlb_buddy_alloc_fail 0 >>> unevictable_pgs_culled 1006521 >>> unevictable_pgs_scanned 0 >>> unevictable_pgs_rescued 997077 >>> unevictable_pgs_mlocked 1319203 >>> unevictable_pgs_munlocked 842471 >>> unevictable_pgs_cleared 470531 >>> unevictable_pgs_stranded 459613 >>> thp_fault_alloc 20263113 >>> thp_fault_fallback 3368635 >>> thp_collapse_alloc 226476 >>> thp_collapse_alloc_failed 17594 >>> thp_file_alloc 0 >>> thp_file_mapped 0 >>> thp_split_page 1159 >>> thp_split_page_failed 3927 >>> thp_deferred_split_page 20348941 >>> thp_split_pmd 53361 >>> thp_split_pud 0 >>> thp_zero_page_alloc 1 >>> thp_zero_page_alloc_failed 0 >>> thp_swpout 0 >>> thp_swpout_fallback 0 >>> balloon_inflate 0 >>> balloon_deflate 0 >>> balloon_migrate 0 >>> swap_ra 0 >>> swap_ra_hit 0 >>> >>> Greets, >>> Stefan >>> >>
