Re: Best COPY Performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



>>>> 1 0 345732 29304 770272 12946764  0  0 16 16428 1192 3105 12  2 85  1
>>>> 1 0 345732 30840 770060 12945480  0  0 20 16456 1196 3151 12  2 84  1
>>>> 1 0 345732 32760 769972 12943528  0  0 12 16460 1185 3103 11  2 86  1
>>
>> iirc, he is running quad opteron 885 (8 cores), so if my math is
>> correct he can split up his process for an easy gain.
>
> Are you saying that I should be able to issue multiple COPY commands
> because my I/O wait is low?  I was under the impression that I am I/O
> bound, so multiple simeoultaneous loads would have a detrimental
> effect ...

The reason I asked how many CPUs was to make sense of the 12% usr CPU time
in the above.  That means you are CPU bound and are fully using one CPU.  So
you aren't being limited by the I/O in this case, it's the CPU.
... snip ...
For now, you could simply split the file in two pieces and load two copies
at once, then watch the same "vmstat 1" for 10 seconds and look at your "bo"
rate.

Significantly higher on average, and a parallel loads were ~30% faster
that a single with index builds (240s vs 340s) and about ~45% (150s vs
230s) without the PK index.  I'll definitely look into the bizgres
java loader.

Thanks!


[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Yosemite]

  Powered by Linux