|
On 04/02/2014 02:36 PM, Adrian Klaver
wrote:
On
04/02/2014 01:14 PM, Rob Sargent wrote:
On 04/02/2014 01:56 PM, Steve Atkins
wrote:
Impatience got the better of me and I
killed the second COPY. This time it had done 54% of the
file in 6.75 hours, extrapolating to roughly 12 hours to do
the whole thing.
That seems rather painfully slow. How exactly are you doing
the bulk load? Are you CPU limited or disk limited?
Have you
readhttp://www.postgresql.org/docs/current/interactive/populate.html
?
Cheers,
Steve
The copy command was pretty vanilla:
copy oldstyle from
'/export/home/rob/share/testload/<file-redacted>'
with delimiter ' ';
I've been to that page, but (as I read
them) none sticks out as a sure
thing. I'm not so worried about the actual performance as I am
with the
relative throughput (sixes so far).
Have you looked at the Postgres logs from that time period to see
if there is anything of interest, say complaining about
checkpoints.
I'm not cpu bound, but I confess I didn't look at io stats
during the
copy runs. I just assume it was pegged :)
Thanks,
Well indeed there are copious LOG/HINT pairs along the lines of
LOG: checkpoints are occurring too frequently ([8 <=
n <=29] seconds apart)
HINT: Consider increasing the configuration parameter
"checkpoint_segments".
and these are during non-load periods. During the COPYs I see
uncountable numbers of
WARNING: pgstat wait timeout
As you guys have probably already concluded, I have some
config/tuning to do.
Cheers,
rjs
|
