On Tue, Mar 9, 2021 at 6:03 PM Tom Lane <tgl@xxxxxxxxxxxxx> wrote:
Radoslav Nedyalkov <rnedyalkov@xxxxxxxxx> writes:
> Occasionally we get bad queries on our db that consume a lot of memory.
> These typically are full joins by mistake or just too large result sets.
> My understanding is these should go to a temp file but apparently memory
> allocation is preferred.
Perhaps the accumulation is happening on the client side? libpq doesn't
have any provision for spilling a result set to disk.
If that's it, you could consider revising your application to read results
row-at-a-time, although that might require a good deal of effort.
https://www.postgresql.org/docs/current/libpq-single-row-mode.html
regards, tom lane
Ah, I named it result set wrongly perhaps.
These are queries , part of a larger ETL function or statement which at the end just write to a table.
The huge join is an intermediate step.
So I'm still wondering.
Thanks and regards,
Rado
