Search Postgresql Archives

Re: pg_restore out of memory

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Tom Lane writes:

What's more, because the line and field buffers are StringInfos that are
intended for reuse across multiple lines/fields, they're not simply made
equal to the exact size of the big field.  They're rounded up to the
next power-of-2, ie, if you've read an 84MB field during the current
COPY IN then they'll be 128MB apiece.  In short, COPY is going to need
508MB of process-local RAM to handle this row.

Of shared memory? I am a little confused,yesterday you said that increasing shared_buffers may be counterproductive. Or you are referring to the OS size?
The OS size is 1.6GB, but today I am going to try increasing kern.maxssiz.
Vivek recommended increasing it

In short, you need a bigger per-process memory allowance.

I wrote a mini python program to copy one of the records that is failing.
The client program is using 475MB with 429MB resident.

The server has been running all night on this single insert.
The server is using 977MB with 491MB resident.
Yesterday I saw it grow as big as 1000MB with 900MB+ resident.


BTW: I think if you were using different client and server encodings
there would be yet a sixth large buffer involved, for the output of
pg_client_to_server.

Using default encoding.

The log file keeps growing with simmilar messages.
Put subset of log at http://public.natserv.net/postgresql-2007-06-19.txt

However the process doesn't crash.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Postgresql Jobs]     [Postgresql Admin]     [Postgresql Performance]     [Linux Clusters]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Postgresql & PHP]     [Yosemite]
  Powered by Linux