On 12/19/2013 03:24 PM, Sergey Konoplev wrote:
2. You are limited with IO I would also suggest you to upgrade your storage in this case.
I think this is the case. If I recall correctly, his setup includes a single RAID-1 for everything, and he only has 32GB of RAM. In fact, the WAL traffic from those inserts alone are likely saturating the write IO, especially if it starts a checkpoint while the load is still going on. I wouldn't want to be around for that.
Even with a fairly selective index, just the fetches necessary to identify the rows and verify the data pages will choke a RAID-1 with almost every query. Any table with several hundred million rows is also too big to fit in cache if any significant portion of it is fetched on a regular basis. The cache turnover is probably extremely high, too.
That workload is just too high for a system of that description. It would be fine for a prototype, development, or possibly a QA system, but if that's intended to be a production resource, it needs more memory and IO.
Also since I can't see part of this conversation and it doesn't seem anyone else mentioned it, the WAL directory *must* be moved to a separate set of disks for a workload of this volume. The amount of writes here will constantly degrade read IO and further increase fetch times.
-- Shaun Thomas OptionsHouse | 141 W. Jackson Blvd. | Suite 500 | Chicago IL, 60604 312-676-8870 sthomas@xxxxxxxxxxxxxxxx ______________________________________________ See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email -- Sent via pgsql-performance mailing list (pgsql-performance@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
