On Sat, 30 Sep 2023 13:40:37 -0700 Adrian Klaver wrote: >On 9/30/23 11:32, pf@xxxxxxxxxxx wrote: >> On Sat, 30 Sep 2023 13:20:14 -0400 Tom Lane wrote: >> > >> As vanilla as it gets... Standard locale (C). The only odd thing that >> happened: a system update the other day installed and started something >> called tracker-miners which I was not happy with: >> https://bugs.mageia.org/show_bug.cgi?id=32340 >> I was quite upset that the distro guys would install/run what they thought >> could be a "cool" tool (as you'll see in the above link). >> >>> 2. There's something wrong with your Postgres installation. >> >> Scary; but that's what I'm suspecting too now... > >The script ran on my installation also. It would seem it is something on >your end. Has there been any issues with your machine and/or database >e.g. crashes or other odd behavior(not counting the tracker-miners update)? As I told Tom, the "test" DB has this issue; the production and test1 DBs are fine; I should have thought to check those first... Sorry for the noise. >FYI, since you are using psycopg2 you might want to use the built in >tools for: > > a) Dynamic SQL > https://www.psycopg.org/docs/sql.html > > b) COPY > https://www.psycopg.org/docs/usage.html#using-copy-to-and-copy-from Thanks for these links... They look very interesting. Someone on this list suggested Practical SQL by Anthony DeBarros which we're working through... He should see sales from all over the US now.... :) We were working with a python script (using INSERT) someone wrote a couple years ago; with my version using COPY[1], imports have gone from 2+ hours to under a minute each for 8M and 33M row files. If the above links help even more that will be amazing!! THANK YOU!! [1] the script I sent is just a test script I threw together to check out COPY features -- only started; but well impressed. >> Understood; I wasn't sure which rabbit hole to go down; but this is >> starting to smell like a DB issue... I'll see about re-installing the >> server and pray the DB is sane... If the above tracker-miners found my >> postgres installation and added something to it; then I will be EXTREMELY >> upset... > > From here: > >https://github.com/GNOME/tracker-miners > >it looks like the software use SQLite for data storage. > >> >> Thanks for confirming my simple SQL runs... >> >>> regards, tom lane >> >> Thanks Tom! >> >> >
