On Tue, Aug 18, 2009 at 1:48 PM, Greg Stark <gsstark@xxxxxxx> wrote:
On Tue, Aug 18, 2009 at 6:39 PM, Michael Clark<codingninja@xxxxxxxxx> wrote:From what you posted earlier it looked like it was turning into about
> But it seems pretty crazy that a 140meg bit of data goes to 1.3 gigs. Does
> that seem a bit excessive?
500M which sounds about right. Presumably either libpq or your code is
holding two copies of it in ram at some point in the process.
From what I saw, stopped at this line in my code running through gdb:
const char *valC = PQgetvalue(result, rowIndex, i);
my mem usage was 300megs. Stepping over this line it went to 1.3 gigs.
Unless there is some way to misconfigure something, I can't think how my code could do that.
I will profile it and see if I can tell who is holding on to that memory.
8.5 will have an option to use a denser hex encoding but it will still
be 2x as large as the raw data.
Sweet!
There are definitely disadvantages. Generally it requires you to know
> I avoided the binary mode because that seemed to be rather confusing when
> having to deal with non-bytea data types. The docs make it sound like
> binary mode should be avoided because what you get back for a datetime
> varies per platform.
what the binary representation of your data types is and they're not
all well documented or guaranteed not to change in the future. I
wouldn't recommend someone try to decode a numeric or a postgres array
for example. And floating point numbers are platform dependent. But
bytea is a case where it seems more natural to use binary than text
representation.
To do something like this, I guess it would be best for my wrapper to being to detect when I have a bytea column in a table and do 2 fetchs, one in text for all other columns, and one in binary for the bytea column. Is this the best way to handle that do you think?
Thanks,
Michael.
