Re: Error: "Out of memory while reading tuples." in pushing table from SAS to PostgreSQL on Mac

[wetter wetterana <wetterana@xxxxxxxxx>]

Apologies, 

I am using SAS, a statistical package/database management system.  SAS has the feature of connecting to a PostgreSQL server.  It does so by assigning what is called a libname (a 'library' connection telling SAS that a particular folder is a data storage). In this assignment, you specify the server address, databasename, password.  I.e. SAS connects to the server and allows you to push data to/read data/manipulate data in the PSQL database.

Here is the code in SAS:

# connect to the server and assign it the name 'psqlibrary'

libname psqlibrary postgres server='xxx.xx.xxx.x' port=5432 

   user=username password='xxxxxxx' database=dbname;

# then append data to the table 'table' in the psql database 

proc append base = psqlibrary.table

data = "">

run;

SAS reports back error messages from the server if it receives such errors.  In my case, I repeatedly get the error described above.  This error is also reported in my terminal app if I log into the database at the same time to monitor things there. I.e. as far as I can tell, the error "Out of memory while reading tuples.; No query has been executed with that handle" stems from PSQL itself. SAS prefixes the error with some specific indicators such as "GLOBAL SYSDBMSG POSTGRES"  or "ERROR: CLI describe error".

Adrian, you are right, PSQL does receive the data. Also not sure what you mean by cursor, I really new to PSQL... sorry!

Help much appreciated!

2014-12-14 10:51 GMT-05:00 Adrian Klaver <adrian.klaver@xxxxxxxxxxx>:

On 12/14/2014 07:44 AM, Andy Colson wrote:

On 12/13/2014 10:03 PM, wetter wetterana wrote:

Hi,

I'm passing rows from SAS to PostgreSQL (I assign a libname and use a
PROC APPEND).  This works fine with smaller tables (~below 1 million
rows).  However, as tables get larger I receive the following error
messages:


"ERROR: CLI describe error: Out of memory while reading tuples.; No
query has been executed with that handle"
and
"GLOBAL SYSDBMSG POSTGRES: Out of memory while reading tuples.; No
query has been executed with that handle
GLOBAL SYSDBRC HY000"

I've tried to change memory settings on the PostgreSQL server, but
can't solve the problem.  As far as I could understand—I'm new to
PostgreSQL ;)—it seems that PostgreSQL want to somehow read
information on the whole table before processing it and this behavior
could eventually be switched off, but I might be wrong here.

FYI:
- I run SAS 9.4 on a windows machine.
- I run PostgreSQL server on a MAC: PostgreSQL 9.3.5 on
x86_64-apple-darwin12.5.0, compiled by Apple LLVM version 5.1, 64-bit
   I've Pgadmin 1.18.1 installed.

Any help would be much appreciated!!

THANKS!


PS: For several reasons, I cannot use the bulkload feature in SAS for
this job.

That error is coming from SAS, not PG.  SAS must have pretty bad
documentation because when I google "GLOBAL SYSDBRC HY000" there is
pretty much nothing.  (Also, I've never used, or even heard of SAS.  At
first I thought you meant serial attached scsi)

SAS is statistical software:

http://www.sas.com/en_us/home.html

Last time I used it mainframes roamed the Earth:)

Does SAS support a cursor of some kind?

-Andy

--
Adrian Klaver
adrian.klaver@xxxxxxxxxxx

--
Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general