Re: Processing very large TEXT columns (300MB+) using C/libpq

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



As an aside any halfway decent optimizer would realize that the results of strlen() are unchanging as long as the contents of what it's passed isn't modified. That's a common enough pattern that it should be checked.

What about buffer size? Are you using a smaller fetch size that results in lots of little packets?

Is there a way to specify a connection that uses compression?

On Fri, Oct 20, 2017 at 10:55 AM, Tom Lane <tgl@xxxxxxxxxxxxx> wrote:
Cory Nemelka <cnemelka@xxxxxxxxx> writes:
> Yes, but I should be able to read them much faster.  The psql client can
> display an 11MB column in a little over a minute, while in C using libpg
> library, it takes over an hour.

Well, of course psql relies on libpq, so it seems unlikely that libpq
itself is where the time is going.  Have you tried applying a profiler?
"perf" or "oprofile" or similar tool ought to pinpoint the culprit
pretty easily.

                        regards, tom lane


--
Sent via pgsql-admin mailing list (pgsql-admin@xxxxxxxxxxxxxx)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux