Re: Bunching "transactions"

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Chris Browne wrote:
> jeandavid8@xxxxxxxxxxx (Jean-David Beyer) writes:
>> But what is the limitation on such a thing? In this case, I am just
>> populating the database and there are no other users at such a time. I am
>> willing to lose the whole insert of a file if something goes wrong -- I
>> would fix whatever went wrong and start over anyway.
>>
>> But at some point, disk IO would have to be done. Is this just a function of
>> how big /pgsql/data/postgresql.conf's shared_buffers is set to? Or does it
>> have to do with wal_buffers and checkpoint_segments?
> 
> I have done bulk data loads where I was typically loading hundreds of
> thousands of rows in as a single transaction, and it is worth
> observing that loading in data from a pg_dump will do exactly the same
> thing, where, in general, each table's data is loaded as a single
> transaction.

I guess a reasonable standard of performance would be that if my initial
population of the database takes only a little longer than a restore of the
database using pg_restore, I am pretty close, and that is good enough. Of
course, the restore depends on how fast my tape drive can pull the tape --
it claims up to 12 MB/sec transfer rate, so it looks as though it will be
tape-limited rather than postgreSQL-limited.
> 
> It has tended to be the case that increasing the number of checkpoint
> segments is helpful, though it's less obvious that this is the case in
> 8.2 and later versions, what with the ongoing changes to checkpoint
> flushing.

I am running postgresql-8.1.9-1.el5 because that is what comes with RHEL5.
I probably will not upgrade until a little while after RHEL7 comes out,
since I hate upgrading.
> 
> In general, this isn't something that typically needs to get tuned
> really finely; if you tune your DB, in general, "pretty big
> transactions" should generally work fine, up to rather large sizes of
> "pretty big."


-- 
  .~.  Jean-David Beyer          Registered Linux User 85642.
  /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org
 ^^-^^ 15:05:01 up 2 days, 7:23, 5 users, load average: 4.11, 4.22, 4.16

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

                http://www.postgresql.org/about/donate

[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Yosemite]

  Powered by Linux