Re: plsql gets "out of memory"

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Sep 2, 2011 at 9:30 PM, Rural Hunter <ruralhunter@xxxxxxxxx> wrote:
> Hi Kevin,
>
> I did another try with following additional changes based on our discussion:
> 1. use the tcp connection
> 2. turn off autovacuum
> 3. turn off full_page_writes
>
> I could import more than 30G data in about 2 hours. That's totally
> acceptable performance to me with the current server capability.  There is a
> minor issue though. I saw a few errors during the import:
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe6272c
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe5272c
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe5272c
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe5272c
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe68e27
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe7272c
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe5272c
> ERROR:  invalid byte sequence for encoding "UTF8": 0xe5a427
>
> My data was exported from an UTF8 MySQL database and my pgsql db is also
> UTF8. I got 8 errors above only with about 3 million records imported. The
> strange thing is, I usually see the problematic SQL output in the log if
> there is any error for that SQL so I have a chance to fix the data manually.
> But for the errors above, I don't see any SQL logged. The pgsql log just
> output error log same as above with no additional info:
> 2011-09-01 11:26:32 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe6272c
> 2011-09-01 11:26:47 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe5272c
> 2011-09-01 11:26:53 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe5272c
> 2011-09-01 11:26:58 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe5272c
> 2011-09-01 11:26:58 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe68e27
> 2011-09-01 11:27:01 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe7272c
> 2011-09-01 11:27:06 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe5272c
> 2011-09-01 11:27:15 CST ERROR:  invalid byte sequence for encoding "UTF8":
> 0xe5a427
>
> What could be the cause of that?

MySQL probably has looser checking of proper UTF-8 encodings.

-- 
Sent via pgsql-admin mailing list (pgsql-admin@xxxxxxxxxxxxxx)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin



[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux