-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/16/2013 08:40 AM, Tom Lane wrote: > "David M. Kaplan" <david.kaplan@xxxxxx> writes: >> Thanks for the help. You have definitely identified the problem, >> but I am still looking for a solution that works for me. I tried >> setting vm.overcommit_memory=2, but this just made the query >> crash quicker than before, though without killing the entire >> connection to the database. I imagine that this means that I >> really am trying to use more memory than the system can handle? > >> I am wondering if there is a way to tell postgresql to flush a >> set of table lines out to disk so that the memory they are using >> can be liberated. > > Assuming you don't have work_mem set to something unreasonably > large, it seems likely that the excessive memory consumption is > inside your PL/R function, and not the fault of Postgres per se. > You might try asking in some R-related forums about how to reduce > the code's memory usage. The two "classic" approaches to this with PL/R are either create a custom aggregate with the PL/R as the final function (i.e. work on one group at a time) or use the SPI cursor functionality within the PL/R function and page your data using a cursor. Not all forms of analysis lend themselves to these approaches, but perhaps yours does. Ultimately I would like to implement a form of R data.frame that does the paging with a cursor transparently for you, but I have not been able to find the time so far. Joe - -- Joe Conway credativ LLC: http://www.credativ.us Linux, PostgreSQL, and general Open Source Training, Service, Consulting, & 24x7 Support -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRlVJdAAoJEDfy90M199hle8gP+wU+GSJ44g26VBBAy3po/E/Y 9+pwxBhJe0x6v5PXtuM8Bzyy4yjlKCgzDj4XdZpEU7SYR+IKj7tWCihqc+Fuk1t1 EjR2VUJwpSMztRvEIqWW8rX/DFGaVYCt89n0neKfKL/XJ5rbqMqQAUPbxMaBtW/p 7EXo8RjVBMYibkvKrjpYJjLTuOTWkQCiXx5hc4HVFN53DYOF46rdFxMYUe5KLYTL mZOnSoV0yrsaPGnxRIY0uzRv7ZTTBmB2o4TIWpTySx2rHNLqAJIT22wl0pfkjksH JYvko3rWhSg7vSf+8RDN6X1eMAXcUO7H2NR5IdOoXEX2bzqTmDBQUjOcb5WR1yUd L5XuT5WYiTpyzU8qAtPEVirwFnEwUN1tR6wDoVsseIWwXUYqSuXtg9qjFNAXZ1Hr 05yxuzexOEzLQNwSXWhsCrLdnndEHrJ6pDlLaUCPVybxwwwW9BfS2fJUz+X63M8x l5DYbyl6q6o2J2bs4UGCTk4r/1Qq/R9pApkWzsckTtF6zl49mzwzPnh5b/JcB+4x u17Te+s3cRGcX09lt7qf9cWkv1uUF/Qw0ntBhW8TY2HYhbWVIEmiZV1HIksXf+nw EBFshWs2/H75OPnhN9YNq3tjCuiR7o/eaZeINfGs2LzGIJvHpcjMDBgFFTES7CYV Y20XukH07h9XcJGTsf0o =TwfD -----END PGP SIGNATURE----- -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
