Re: pg_dump out of memory for large table with LOB

[Adrien Nayrat <adrien.nayrat@xxxxxxxxxxxx>]

Hello,

On 11/10/18 12:49 AM, Jean-Marc Lessard wrote:
> The dumped table is 0.5TB, 17 million rows and LOB uses about 99% of the space.
> 

If I understand, you have 17 million Large Object?

I do not recall exactly and maybe I am wrong. But it seems pg_dump has to
allocate memory for each object to dump :
addBoundaryDependencies:

    for (i = 0; i < numObjs; i++)
[...]

case DO_BLOB_DATA:
    /* Data objects: must come between the boundaries */
    addObjectDependency(dobj, preDataBound->dumpId);
    addObjectDependency(postDataBound, dobj->dumpId);
    break;

addObjectDependency:

[...]
                pg_malloc(dobj->allocDeps * sizeof(DumpId));


With 17 million LO, it could eat lot of memory ;)

Attachment: signature.asc
Description: OpenPGP digital signature