Re: libpq - lack of support to set the fetch size

[John R Pierce <pierce@xxxxxxxxxxxx>]

On 3/9/2014 6:43 AM, matshyeq wrote:

Hello,

I've found an issue when tried to implement fetching rows from big table (2mln rows) in my app.

Basically I don't find an elegant and easy way (other than always use cursors) to limit the number of rows returned.

This causes my application to break due to the excessive memory consumption.

I'm using Perl and DBD::Pg library but contacted maintainer who actually pointed out this is an issue that goes much deeper (libpq):

"Unfortunately, this is a limitation in the underlying driver (libpq) rather than DBD::Pg itself. There have been talks over the years of supporting this, but nothing concrete yet. Your best bet would be to ask about this on the Postgres lists"

in addition to what the others suggested, you can use a CURSOR to read through results in arbitrary sized blocks.

-- 
john r pierce                                      37N 122W
somewhere on the middle of the left coast