Hi Pierre,
First of all , I Thank all for sharing the information on this Issue.
On Thu, Mar 25, 2010 at 11:44 PM, Pierre C <lists@xxxxxxxxxx> wrote:
If you do large transactions, which emits large quantities of xlog, be aware that while the previous xlog segment is being fsynced, no new writes happen to the next segment. If you use large wal_buffers (more than 16 MB) these buffers can absorb xlog data while the previous segment is being fsynced, which allows a higher throughput. However, large wal_buffers also mean the COMMIT of small transactions might find lots of data in the buffers that noone has written/synced yet, which isn't good. If you use dedicated spindle(s) for the xlog, you can set the walwriter to be extremely aggressive (write every 5 ms for instance) and use fdatasync. This way, at almost every rotation of the disk, xlog gets written. I've found this configuration gives increased throughput, while not compromising latency, but you need to test it for yourself, it depends on your whole system.
Small testing is done from my end. I have created a "test" table with one row and done insertion into it(10,00,000- rows). I have turned off fsync and syncronous_commit. I saw there is fast insert if i do so, but if i turn it on then there is latency.
Before fsync / syncronous_commit on
============================
postgres=# explain analyze insert into test values(generate_series(1,1000000));
QUERY PLAN
---------------------------------------------------------------------------------------------
Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.015..6293.674 rows=1000000 loops=1)
Total runtime: 37406.012 ms
(2 rows)
QUERY PLAN
---------------------------------------------------------------------------------------------
Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.015..6293.674 rows=1000000 loops=1)
Total runtime: 37406.012 ms
(2 rows)
After fsync/syncronous_commit off
=========================
postgres=# explain analyze insert into test values(generate_series(1,1000000));
QUERY PLAN
---------------------------------------------------------------------------------------------
Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.154..5801.584 rows=1000000 loops=1)
Total runtime: 29378.626 ms
(2 rows)
QUERY PLAN
---------------------------------------------------------------------------------------------
Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.154..5801.584 rows=1000000 loops=1)
Total runtime: 29378.626 ms
(2 rows)
I request to know here is, what would be xlog files with wal_buffer. Does xlog will recycle or grow in creating one more for this particular transaction. Could you explain here, when wal_buffer is 64kb which is very small, and everything is in xlog files written, so wt happens if we increase the wal_buffer here?
Regards
Raghav
