Re: How to monitor Postgres real memory usage

[徐志宇徐 <xuzhiyuster@xxxxxxxxx>]

Hi Justin

  Thanks for your update. I collect those logs for your reference.

  I also collect this file /etc/sysctl.conf and PG logs on the attachment. (memory_issue.tar.gz).

 

  when the memory issue occur. The log directory will generate a new log file.

 postgresql-2022-05-27_000000.log. Usually only one file to record log information. 

 For example:postgresql-2022-05-27_000000.csv

 The log which named with *.log will contain a lot of infomation about memory detail.

What's the output of these commands?
tail /proc/sys/vm/overcommit_*

[root@bl4n3icpms vm]# tail /proc/sys/vm/overcommit_*
==> /proc/sys/vm/overcommit_kbytes <==
0

==> /proc/sys/vm/overcommit_memory <==
2

==> /proc/sys/vm/overcommit_ratio <==
60



tail /proc/sys/vm/nr_*hugepages /proc/cmdline

==> /proc/sys/vm/nr_hugepages <==
0

==> /proc/sys/vm/nr_overcommit_hugepages <==
0

==> /proc/cmdline <==
BOOT_IMAGE=/vmlinuz-3.10.0-957.27.2.el7.x86_64 root=/dev/mapper/rootvg-lv_root ro crashkernel=auto rd.lvm.lv=rootvg/lv_root rd.lvm.lv=rootvg/lv_swap rhgb quiet LANG=en_US.UTF-8


cat /proc/meminfo

[root@bl4n3icpms vm]# cat /proc/meminfo
MemTotal:       16266368 kB
MemFree:          203364 kB
MemAvailable:    7823244 kB
Buffers:            3272 kB
Cached:         12978488 kB
SwapCached:            0 kB
Active:         10456284 kB
Inactive:        5042108 kB
Active(anon):    6008156 kB
Inactive(anon):  1738892 kB
Active(file):    4448128 kB
Inactive(file):  3303216 kB
Unevictable:       11292 kB
Mlocked:           11292 kB
SwapTotal:       2097148 kB
SwapFree:        2097148 kB
Dirty:                20 kB
Writeback:             0 kB
AnonPages:       2527924 kB
Mapped:          4527276 kB
Shmem:           5226728 kB
Slab:             268304 kB
SReclaimable:     206876 kB
SUnreclaim:        61428 kB
KernelStack:        4576 kB
PageTables:       106924 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    11856968 kB
Committed_AS:   10488212 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       37468 kB
VmallocChunk:   34359695100 kB
HardwareCorrupted:     0 kB
AnonHugePages:    124928 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      143208 kB
DirectMap2M:     6148096 kB
DirectMap1G:    12582912 kB



uname -a

Linux bl4n3icpms.lenovo.com 3.10.0-957.27.2.el7.x86_64 #1 SMP Mon Jul 29 17:46:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux


[root@bl4n3icpms vm]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.accept_redirects = 0
# edit for pg database used#
kernel.shmmax=16656777216
kernel.shmall=4066596
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.sem = 250     32000   32      128
kernel.pid_max=131072
vm.overcommit_memory=2
vm.overcommit_ratio=60
vm.swappiness=0
# edit for pg database used#

Justin Pryzby <pryzby@xxxxxxxxxxxxx> 于2022年5月27日周五 07:35写道：

On Fri, May 27, 2022 at 01:39:15AM +0800, 徐志宇徐 wrote:
> Hi Justin
>
>   Thanks for you explaination.
>
>   > > What postgres version ?
>   > > How was it installed ?  From souce?  From a package ?
>   I am using Postgres 11.1 .It's installed by package.

This is quite old, and missing ~4 years of bugfixes.

What's the output of these commands?
tail /proc/sys/vm/overcommit_*
tail /proc/sys/vm/nr_*hugepages /proc/cmdline
cat /proc/meminfo
uname -a

--
Justin

Attachment: memory_issue.tar.gz
Description: GNU Zip compressed data