Re: COPY from STDIN vs file with large CSVs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 1/8/20 10:54 AM, Wells Oliver wrote:
I have a CSV that's ~30GB. Some 400m rows. Would there be a meaningful performance difference to run COPY from STDIN using: cat f.csv | psql "COPY .. FROM STDIN WITH CSV" versus just doing "COPY ... FROM 'f.csv' WITH CSV"?

Thanks. It took about four hours to copy one and I felt that was a little much.

catting the file starts another process, and opens a pipe.  That can't be faster than "COPY ... FROM ... WITH CSV".

pg_bulkload (which might be in your repository) is probably what you really want.

--
Angular momentum makes the world go 'round.





[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux