We have faced in issue in our Postgresql 9.5.13 cluster. Inserts into btree index are too slow when strings contain Thai characters.Test script and results are in the attachment. Test shows that insert Thai string into index is more than 60x times slower than Chinese or Russian, for example. Tracing with perf showed that problem is in strcoll_l() libc function (see thai-slow.svg). This function is used when locale is different from 'C'. For 'C' locale just simple comparison is used (see thai-fast.graph) and performance is OK. Of course, I googled and thought that it is a bug in glibc (https://sourceware.org/bugzilla/show_bug.cgi?id=18441), but when I tried previous version of glibc (2.19 and 2.13) I found out that it still reproduced. I know that I can upgrade PostgreSQL to 10 and user libicu for string comparison but is there any way to fix that in PostgreSQL 9.5.13? P.S. I can provide COLLATE "C" for this column during its creation but it looks a little bit tricky. -- With best regards, Andrey Zhidenkov
\timing on Timing is on. \set num_rows 100000 drop table if exists test_thai; DROP TABLE Time: 36.125 ms create table test_thai(id serial, data text); CREATE TABLE Time: 29.668 ms insert into test_thai (data) select 'อัญประกาศ' || i::text from generate_series(1,:num_rows) as t(i); INSERT 0 100000 Time: 151.709 ms truncate test_thai ; TRUNCATE TABLE Time: 16.857 ms create index test_thai_data on test_thai(data); CREATE INDEX Time: 12.381 ms insert into test_thai (data) select 'русский' || i::text from generate_series(1,:num_rows) as t(i); INSERT 0 100000 Time: 676.143 ms truncate test_thai ; TRUNCATE TABLE Time: 26.376 ms insert into test_thai (data) select '汉字/漢字' || i::text from generate_series(1,:num_rows) as t(i); INSERT 0 100000 Time: 1161.516 ms (00:01.162) truncate test_thai ; TRUNCATE TABLE Time: 22.561 ms insert into test_thai (data) select 'อัญประกาศ' || i::text from generate_series(1,:num_rows) as t(i); INSERT 0 100000 Time: 58234.397 ms (00:58.234)
Attachment:
test.sql
Description: application/sql
Attachment:
thai-slow.svg
Description: image/svg
Attachment:
thai-fast.svg
Description: image/svg