On 5/22/20 5:37 AM, Nico De Ranter wrote:
Hi all,
Postgres version: 9.5
OS: Ubuntu 18.04.4
I have a 144GB Bacula database that crashes the postgres daemon when I
try to do a pg_dump.
At some point the server ran out of diskspace for the database storage.
I expanded the lvm and rebooted the server. It seemed to work fine,
however when I try to dump the bacula database the postgres daemon dies
after about 37GB.
I tried copying the database to another machine and upgrading postgres
to 11 using pg_upgrade. The upgrade seems to work but I still get
exactly the same problem when trying to dump the database.
postgres@core4:~$ pg_dumpall --cluster 11/main --file=dump.sql
pg_dump: Dumping the contents of table "file" failed: PQgetCopyData()
failed.
pg_dump: Error message from server: server closed the connection
unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
pg_dump: The command was: COPY public.file (fileid, fileindex, jobid,
pathid, filenameid, deltaseq, markid, lstat, md5) TO stdout;
pg_dumpall: pg_dump failed on database "bacula", exiting
What happens if you try to dump just this table?
Something along lines of:
pg_dump -t file -d some_db -U some_user
Have you looked at the system logs to see if it is the OS killing the
process?
In the logs I see:
2020-05-22 14:23:30.649 CEST [12768] LOG: server process (PID 534) was
terminated by signal 11: Segmentation fault
2020-05-22 14:23:30.649 CEST [12768] DETAIL: Failed process was
running: COPY public.file (fileid, fileindex, jobid, pathid, filenameid,
deltaseq, markid, lstat, md5) TO stdout;
2020-05-22 14:23:30.651 CEST [12768] LOG: terminating any other active
server processes
2020-05-22 14:23:30.651 CEST [482] WARNING: terminating connection
because of crash of another server process
2020-05-22 14:23:30.651 CEST [482] DETAIL: The postmaster has commanded
this server process to roll back the current transaction and exit,
because another server process exited abnormally and possibly corrupted
shared memory.
2020-05-22 14:23:30.651 CEST [482] HINT: In a moment you should be able
to reconnect to the database and repeat your command.
2020-05-22 14:23:30.652 CEST [12768] LOG: all server processes
terminated; reinitializing
2020-05-22 14:23:30.671 CEST [578] LOG: database system was
interrupted; last known up at 2020-05-22 14:15:19 CEST
2020-05-22 14:23:30.809 CEST [578] LOG: database system was not
properly shut down; automatic recovery in progress
2020-05-22 14:23:30.819 CEST [578] LOG: redo starts at 197/D605EA18
2020-05-22 14:23:30.819 CEST [578] LOG: invalid record length at
197/D605EA50: wanted 24, got 0
2020-05-22 14:23:30.819 CEST [578] LOG: redo done at 197/D605EA18
2020-05-22 14:23:30.876 CEST [12768] LOG: database system is ready to
accept connections
2020-05-22 14:29:07.511 CEST [12768] LOG: received fast shutdown request
Any ideas how to fix or debug this?
Nico
--
Nico De Ranter
Operations Engineer
T. +32 16 38 72 10
<http://www.esaturnus.com>
<http://www.esaturnus.com>
eSATURNUS
Philipssite 5, D, box 28
3001 Leuven – Belgium
T. +32 16 40 12 82
F. +32 16 40 84 77
www.esaturnus.com <http://www.esaturnus.com>
** <http://www.esaturnus.com/>
*For Service & Support :*
Support Line Belgium: +32 2 2009897
Support Line International: +44 12 56 68 38 78
Or via email : medical.services.eu@xxxxxxxx
<mailto:medical.services.eu@xxxxxxxx>
--
Adrian Klaver
adrian.klaver@xxxxxxxxxxx
