On 10-10-27 10:00 AM, Divakar Singh wrote:
Hi Steve and other friends,
Some information you would be interested in:
I did some further tests using libpq in my code.
I used a stored proc to insert 100 thousand rows in a table, it took 25
sec (almost same as time taken by Oracle PL/SQL and OCI interface).
Same inserts through libpq take 70 seconds.
I am inserting all records in a single transaction.
So, the problem seems to be optimization of usage of libpq in my code.
I am attaching my code below.
Is any optimization possible in this?
Do prepared statements help in cutting down the insert time to half for
this kind of inserts? One of the major problems with libpq usage is lack
of good documentation and examples.
I could not get any good example of prepared stmt usage anywhere.
Yes using prepared statements should make this go faster, but your best
bet might be to use the COPY command. I don't have a PQprepare example
handy though we probably should add one to the docs.
The copy command would be used similar to
PQexec(conn,"COPY TO aaaa (a,b,c,d,e,f,g,h,j,k,l,m,n,p) FROM STDIN WITH
(DELIMITER ',') ");
for(; i < howmany;i++)
{
sprintf(query,"67,'ABRAKADABRA', 'ABRAKADABRA', 'ABRAKADABRA', '1-Dec
2010', 'ABRAKADABRA', 'ABRAKADABRA', 'ABRAKADABRA', '1-Dec-2010',
99999, 99999, %d, 9999,
'ABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRA'\n",
i);
res = PQputCopyData(conn,query,strlen(query);
}
PQputCopyEnd(conn,NULL);
I have not actually tried the above code snippet, it is just to give you
the general idea.
You call PQexec with the COPY command outside the loop then at each loop
iteration you call PQputCopyData with some of the data that gets passed
to the server.
You can combine multiple lines on a single PQputCopyData call if you want.
http://www.postgresql.org/docs/9.0/interactive/sql-copy.html
http://www.postgresql.org/docs/9.0/interactive/libpq-copy.html
char query[1024]={0};
for (; i<howmany;i++ )
{
sprintf (query, "INSERT INTO aaaa(a, b, c, d, e, f, g, h, j, k, l, m, n,
p) VALUES (67, 'ABRAKADABRA', 'ABRAKADABRA', 'ABRAKADABRA',
'1-Dec-2010', 'ABRAKADABRA', 'ABRAKADABRA', 'ABRAKADABRA', '1-Dec-2010',
99999, 99999, %d, 9999,
'ABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRAABRAKADABRA')",
i);
res = PQexec(conn, query);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
cout<<"error at iteration "<<i<<":"<<PQresultErrorMessage(res)<<endl;
PQclear(res);
break;
}
//PQclear(res);
}
/* close the portal ... we don't bother to check for errors ... */
/*res = PQexec(conn, "CLOSE myportal");
PQclear(res);*/
<snip>
--
Sent via pgsql-performance mailing list (pgsql-performance@xxxxxxxxxxxxxx)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
