Hi Adrian & Tom, On Sat, 30 Sep 2023 15:57:32 -0700 Adrian Klaver wrote: >On 9/30/23 14:54, pf@xxxxxxxxxxx wrote: >> On Sat, 30 Sep 2023 13:40:37 -0700 Adrian Klaver wrote: >> >>> On 9/30/23 11:32, pf@xxxxxxxxxxx wrote: > >> >> As I told Tom, the "test" DB has this issue; the production and test1 DBs >> are fine; I should have thought to check those first... Sorry for the >> noise. > >Still there was an issue with a database. Did you track down what was >wrong with "test"? Background: it's one of 18 databases on: /dev/nvme1n1p1 3.6T 1.3T 2.2T 38% /mnt/work Tried using a never before used table name; same error. I wondered if an old SEQUENCE might be the cause; nope. OK.. after: * back up a table * try import -- fails * drop backed-up table rinse and repeat until the test database is empty.. Still fails. Backed up the empty db (attached). I see the string "SET client_encoding = 'WIN1252';" in the dump -- some files come from a remote colleague; but this has never been an issue before... Next, I was going to connect to test1 with SQL-workbench/J when I saw this: "Cannot invoke "workbench.storage.RowData.getValue(int)" because "<local4>" is null" on the SQL Source tab. Emptied test1. Imported the table just fine. Dropped it. Dumped the db (attached) which is 2 bytes smaller. "test1" appears 4 times, so this dump should be 2 bytes larger, not smaller; it also says 'UTF8' instead of 'WIN1252'. Lots of other diffs... Hope there's something in there you guys can make sense of... Regards, Pierre >> We were working with a python script (using INSERT) someone wrote a couple >> years ago; with my version using COPY[1], imports have gone from 2+ hours >> to under a minute each for 8M and 33M row files. If the above links help >> even more that will be amazing!! THANK YOU!! > >The move to COPY will have accounted for the speed up. Using the >psycopg2 COPY methods probably won't make a material difference, I just >find them easier to use. Also in the new psycopg(3) there are more ways >to use them per: > > https://www.psycopg.org/psycopg3/docs/basic/copy.html. > > > >
Attachment:
test_empty.backup
Description: Binary data
Attachment:
test1_empty.backup
Description: Binary data
