On 9/30/23 14:54, pf@xxxxxxxxxxx wrote:
On Sat, 30 Sep 2023 13:40:37 -0700 Adrian Klaver wrote:
On 9/30/23 11:32, pf@xxxxxxxxxxx wrote:
As I told Tom, the "test" DB has this issue; the production and test1 DBs
are fine; I should have thought to check those first... Sorry for the
noise.
Still there was an issue with a database. Did you track down what was
wrong with "test"?
We were working with a python script (using INSERT) someone wrote a couple
years ago; with my version using COPY[1], imports have gone from 2+ hours
to under a minute each for 8M and 33M row files. If the above links help
even more that will be amazing!! THANK YOU!!
The move to COPY will have accounted for the speed up. Using the
psycopg2 COPY methods probably won't make a material difference, I just
find them easier to use. Also in the new psycopg(3) there are more ways
to use them per:
https://www.psycopg.org/psycopg3/docs/basic/copy.html.
--
Adrian Klaver
adrian.klaver@xxxxxxxxxxx
