I am in the process of converting an old system to a new system where I have chosen to use postgres in stead of a home grow system based on b-trees. The system receives 2650 message a total of 10Mbytes of data per 15 minutes this information have to be store in 4 tables in the database. Some of the messages are updates to existing data and the result is: 1038 records in table 1 size per record 282 bytes 2074 records in table 2 size per record 36 bytes 21488 records in table 3 size per record 60 bytes 25676 records in table 4 size per record 42 bytes. Without indexes this should be something like 2,7Mbytes/15min. The disk speed is as follows taken from hdparm -Tt /dev/hda: Timing buffer-cache reads: 128 MB in 0.89 seconds =143.82 MB/sec Timing buffered disk reads: 64 MB in 1.45 seconds = 44.14 MB/sec My problem is that the postmaster process is using between 70-90% of the CPU time. (seen using top) and the total wall time with nothing else running. It takes 6 minutes to ingest the data with an empty database and 25 minutes (wall time) if all the data is already in the database. The processing is done as follows: 1. Start transaction 2. check if message is in table 1 and if so delete records from table 1(1 row),2(2 rows),3(30 rows),4(50 rows). 3. split the message and insert the date in table 1(1 row),2(2 rows),3(30 rows),4(50 rows) 4. end transaction I need some suggestions so that I can ingest the data faster. with the same computer and disks. We are running redhat 7.2 on the server and I am using the c interface to write the data to the server. Thanks for any help, Thor. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
