RE: [EXTERNAL] Re: Inserts and bad performance

["Godfrin, Philippe E" <Philippe.Godfrin@xxxxxxx>]

The notion of COPY blocks and asynchronously is very interesting

 

From: Gavin Roy <gavinr@xxxxxxxxxx>
Sent: Wednesday, November 24, 2021 1:50 PM
To: Godfrin, Philippe E <Philippe.Godfrin@xxxxxxx>
Cc: pgsql-general@xxxxxxxxxxxxxxxxxxxx
Subject: [EXTERNAL] Re: Inserts and bad performance

 

 

 

 

On Wed, Nov 24, 2021 at 2:15 PM Godfrin, Philippe E <Philippe.Godfrin@xxxxxxx> wrote:

Greetings

I am inserting a large number of rows, 5,10, 15 million. The python code commits every 5000 inserts. The table has partitioned children.

 

On the Python client side, if you're using psycopg, you should consider using using COPY instead of INSERT if you're not:

 

https://www.psycopg.org/psycopg3/docs/basic/copy.html#copy

 

And if using psycopg2, execute_batch might be of value:

 

https://www.psycopg.org/docs/extras.html?highlight=insert#psycopg2.extras.execute_batch

 

Regards,

 

Gavin