Performance problems with Thai language

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



We have faced in issue in our Postgresql 9.5.13 cluster. Inserts into
btree index are too slow when strings contain Thai characters.Test
script and results are in the attachment. Test shows that insert Thai
string into index is more than 60x times slower than Chinese or
Russian, for example. Tracing with perf showed that problem is in
strcoll_l() libc function (see thai-slow.svg). This function is used
when locale is different from 'C'. For 'C' locale just simple
comparison is used (see thai-fast.graph) and performance is OK.
Of course, I googled and thought that it is a bug in glibc
(https://sourceware.org/bugzilla/show_bug.cgi?id=18441), but when I
tried previous version of glibc (2.19 and 2.13) I found out that it
still reproduced. I know that I can upgrade PostgreSQL to 10 and user
libicu for string comparison but is there any way to fix that in
PostgreSQL 9.5.13?

P.S. I can provide COLLATE "C" for this column during its creation but
it looks a little bit tricky.

-- 
With best regards, Andrey Zhidenkov
\timing on
Timing is on.
\set num_rows 100000
drop table if exists test_thai;
DROP TABLE
Time: 36.125 ms
create table test_thai(id serial, data text);
CREATE TABLE
Time: 29.668 ms
insert into test_thai (data) select 'อัญประกาศ' || i::text from generate_series(1,:num_rows) as t(i);
INSERT 0 100000
Time: 151.709 ms
truncate test_thai ;
TRUNCATE TABLE
Time: 16.857 ms
create index test_thai_data on test_thai(data);
CREATE INDEX
Time: 12.381 ms
insert into test_thai (data) select 'русский' || i::text from generate_series(1,:num_rows) as t(i);
INSERT 0 100000
Time: 676.143 ms
truncate test_thai ;
TRUNCATE TABLE
Time: 26.376 ms
insert into test_thai (data) select '汉字/漢字' || i::text from generate_series(1,:num_rows) as t(i);
INSERT 0 100000
Time: 1161.516 ms (00:01.162)
truncate test_thai ;
TRUNCATE TABLE
Time: 22.561 ms
insert into test_thai (data) select 'อัญประกาศ' || i::text from generate_series(1,:num_rows) as t(i);
INSERT 0 100000
Time: 58234.397 ms (00:58.234)

Attachment: test.sql
Description: application/sql

Attachment: thai-slow.svg
Description: image/svg

Attachment: thai-fast.svg
Description: image/svg


[Postgresql General]     [Postgresql PHP]     [PHP Users]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Yosemite]

  Powered by Linux