Hi Julien,
thanks for answering me.
The only extension I use is repmgr.
I've tried to use gdb to see something (I don't know if i use it
correctly) , below the backtarce :
[16:03:13]root@db13:/tmp$ gdb -q -c /tmp/core
/usr/lib/postgresql/12/bin/postgres
Reading symbols from /usr/lib/postgresql/12/bin/postgres...(no debugging
symbols found)...done.
[New LWP 177990]
warning: Unexpected size of section `.reg-xstate/177990' in core file.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `postgres: 12/main: supervision neteven2
localhost(34868) SELECT '.
Program terminated with signal SIGSEGV, Segmentation fault.
warning: Unexpected size of section `.reg-xstate/177990' in core file.
#0 0x00005636d2d844f1 in equalTupleDescs ()
(gdb)
(gdb) bt
#0 0x00005636d2d844f1 in equalTupleDescs ()
#1 0x00005636d31a65cf in ?? ()
#2 0x00005636d31b5fd3 in hash_search_with_hash_value ()
#3 0x00005636d31a83b1 in assign_record_type_typmod ()
#4 0x00005636d31b4855 in ?? ()
#5 0x00005636d31b4b43 in get_expr_result_type ()
#6 0x00005636d31b4b7b in get_expr_result_tupdesc ()
#7 0x00005636d2e8bcce in get_rte_attribute_is_dropped ()
#8 0x00005636d303fc7a in AcquireRewriteLocks ()
#9 0x00005636d304039e in ?? ()
#10 0x00005636d3043aa2 in QueryRewrite ()
#11 0x00005636d307de90 in ?? ()
#12 0x00005636d307df70 in pg_analyze_and_rewrite ()
#13 0x00005636d307e68f in ?? ()
#14 0x00005636d30804ad in PostgresMain ()
#15 0x00005636d2d73f00 in ?? ()
#16 0x00005636d3006f89 in PostmasterMain ()
#17 0x00005636d2d75128 in main ()
(gdb) cont
The program is not being run.
thomas
Le 21/08/2020 à 14:34, Julien Rouhaud a écrit :
On Fri, Aug 21, 2020 at 2:25 PM Thomas SIMON <tsimon@xxxxxxxxxxx> wrote:
Hi all,
I just had strange behavior on my postgresql instance, with postgresql
auto restart
Looking for logs, I've found a segfault in kern.log
[12:24:09]root@db12:~# cat /var/log/kern.log
2020-08-21T12:00:01.436378+02:00 db12 kernel: postgres[177990]: segfault
at 0 ip 00005636d2d844f1 sp 00007fff4fa69910 error 4 in
postgres[5636d2cb7000+775000]
I've also enabled core dump, file output is :
[12:24:13]root@db12:~# file /data/postgresql/12/main/core
/data/postgresql/12/main/core: ELF 64-bit LSB core file x86-64, version
1 (SYSV), SVR4-style, from 'postgres: 12/main: supervision neteven2
localhost(34868) SELECT', real uid: 110, effective uid: 110, real gid:
114, effective gid: 114, execfn: '/usr/lib/postgresql/12/bin/postgres',
platform: 'x86_64'
In logs , I have these messages
2020-08-21 12:00:01.451 CEST [274137]: [299-1] user=,db=,app=,client=
LOG: server process (PID 177990) was terminated by signal 11:
Segmentation fault
2020-08-21 12:00:01.451 CEST [274137]: [300-1] user=,db=,app=,client=
DETAIL: Failed process was running: SELECT usename,count(*) FROM
pg_stat_activity WHERE pid != pg_backend_pid() GROUP BY usename ORDER BY 1
..
2020-08-21 12:00:02.776 CEST [274137]: [302-1] user=,db=,app=,client=
LOG: archiver process (PID 274215) exited with exit code 1
2020-08-21 12:00:02.774 CEST [274214]: [1-1] user=,db=,app=,client=
WARNING: terminating connection because of crash of another server process
2020-08-21 12:00:02.774 CEST [274214]: [2-1] user=,db=,app=,client=
DETAIL: The postmaster has commanded this server process to roll back
the current transaction and exit, because another s
erver process exited abnormally and possibly corrupted shared memory.
2020-08-21 12:00:02.774 CEST [274214]: [3-1] user=,db=,app=,client=
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
(many times until full restart)
I'm on 12.3 version, on a dedicated host on prem.
Note that version 12.4 is now available, however I don't see any relevant fix.
root@db12:~# dpkg -l | grep postgresql
ii pgdg-keyring 2018.2 all
keyring for apt.postgresql.org
ii postgresql-12 12.3-1.pgdg90+1 amd64
object-relational SQL database, version 12 server
ii postgresql-12-repmgr 5.1.0-1.stretch+1
amd64 replication manager for PostgreSQL 12
ii postgresql-client-12 12.3-1.pgdg90+1
amd64 front-end programs for PostgreSQL 12
ii postgresql-client-common 215.pgdg90+1
all manager for multiple PostgreSQL client versions
ii postgresql-common 215.pgdg90+1
all PostgreSQL database-cluster manager
ii postgresql-server-dev-12 12.3-1.pgdg90+1
amd64 development files for PostgreSQL 12 server-side programming
Could you please help me to find what is the root cause ? for
This is unfortunately not enough information to find the root issue.
Do you have any custom extension? Is there any chance you can get a
backtrace of the generated coredump? See
https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD#Getting_a_trace_from_a_randomly_crashing_backend
for more details on how to do that.
