pg_upgrade fails saying function unaccent(text) doesn't exist

[Gunnlaugur Thor Briem <gunnlaugur@xxxxxxxxx>]

Hi,

I'm trying to upgrade an old PG 9.4 cluster to PG 10, and it's failing on creating an index that uses the unaccent(text) function.

That function is part of the unaccent extension, which is installed in the old DB cluster. I expect pg_upgrade to create that extension as part of the upgrade. It does create other extensions that are installed in the old DB cluster. I don't get why this one isn't included.

Here are the commands I run, and their output:

----- snip -----

$ sudo rm -rf /opt/local/var/db/postgresql10/defaultdb

$ sudo mkdir -p /opt/local/var/db/postgresql10/defaultdb

$ sudo chown postgres:postgres /opt/local/var/db/postgresql10/defaultdb

$ sudo su postgres -c '/opt/local/lib/postgresql10/bin/initdb -D /opt/local/var/db/postgresql10/defaultdb --locale en_US.UTF-8'
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /opt/local/var/db/postgresql10/defaultdb ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    /opt/local/lib/postgresql10/bin/pg_ctl -D /opt/local/var/db/postgresql10/defaultdb -l logfile start

$ sudo su postgres -c '/opt/local/lib/postgresql10/bin/pg_upgrade --old-bindir /opt/local/lib/postgresql94/bin --old-datadir /opt/local/var/db/postgresql94/defaultdb --new-bindir /opt/local/lib/postgresql10/bin --new-datadir /opt/local/var/db/postgresql10/defaultdb'Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for invalid "unknown" user columns                 ok
Checking for roles starting with "pg_"                      ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok

If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.

Performing Upgrade
------------------
Analyzing all rows in the new cluster                       ok
Freezing all rows in the new cluster                        ok
Deleting files from new pg_xact                             ok
Copying old pg_clog to new server                           ok
Setting next transaction ID and epoch for new cluster       ok
Deleting files from new pg_multixact/offsets                ok
Copying old pg_multixact/offsets to new server              ok
Deleting files from new pg_multixact/members                ok
Copying old pg_multixact/members to new server              ok
Setting next multixact ID and offset for new cluster        ok
Resetting WAL archives                                      ok
Setting frozenxid and minmxid counters in new cluster       ok
Restoring global objects in the new cluster                 ok
Restoring database schemas in the new cluster
  dm_test
*failure*

Consult the last few lines of "pg_upgrade_dump_409041.log" for
the probable cause of the failure.
Failure, exiting

----- snip -----

And here is that pg_upgrade_dump_409041.log file:

----- snip -----

command: "/opt/local/lib/postgresql10/bin/pg_dump" --host /private/tmp --port 50432 --username postgres --schema-only --quote-all-identifiers --binary-upgrade --format=custom  --file="pg_upgrade_dump_409041.custom" 'dbname=dm_test' >> "pg_upgrade_dump_409041.log" 2>&1


command: "/opt/local/lib/postgresql10/bin/pg_restore" --host /private/tmp --port 50432 --username postgres --exit-on-error --verbose --dbname 'dbname=dm_test' "pg_upgrade_dump_409041.custom" >> "pg_upgrade_dump_409041.log" 2>&1
pg_restore: connecting to database for restore
pg_restore: creating pg_largeobject "pg_largeobject"
pg_restore: creating pg_largeobject_metadata "pg_largeobject_metadata"
pg_restore: creating SCHEMA "acl_admin"
pg_restore: creating COMMENT "SCHEMA "acl_admin""
pg_restore: creating SCHEMA "public"
pg_restore: creating COMMENT "SCHEMA "public""
pg_restore: creating EXTENSION "fuzzystrmatch"
pg_restore: creating COMMENT "EXTENSION "fuzzystrmatch""
pg_restore: creating EXTENSION "hstore"
pg_restore: creating COMMENT "EXTENSION "hstore""
pg_restore: creating EXTENSION "sslinfo"
pg_restore: creating COMMENT "EXTENSION "sslinfo""
pg_restore: creating TYPE "public.dataimport_job_state"
pg_restore: creating TYPE "public.dblink_pkey_results"...

[...]

pg_restore: creating INDEX "public.ix_semantic_mapping_lower_title"
pg_restore: creating INDEX "public.ix_semantic_mapping_normalize_title"
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 5240; 1259 427215 INDEX ix_semantic_mapping_normalize_title dm_admin
pg_restore: [archiver (db)] could not execute query: ERROR:  function unaccent(text) does not exist
LINE 2:   SELECT lower(unaccent(btrim(regexp_replace($1, '\s+', ' ',...
                       ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.
QUERY: 
  SELECT lower(unaccent(btrim(regexp_replace($1, '\s+', ' ', 'g'), ' "')))

CONTEXT:  SQL function "semantic_normalize" during inlining
    Command was:
-- For binary upgrade, must preserve pg_class oids
SELECT pg_catalog.binary_upgrade_set_next_index_pg_class_oid('427215'::pg_catalog.oid);

CREATE INDEX "ix_semantic_mapping_normalize_title" ON "public"."semantic_mapping" USING "btree" ("public"."semantic_normalize"("title"));

----- snip -----

These occurrences of unaccent at the end are the only ones in the file:

$ sudo grep -n unaccent  pg_upgrade_dump_409041.log g
1713:pg_restore: [archiver (db)] could not execute query: ERROR:  function unaccent(text) does not exist
1714:LINE 2:   SELECT lower(unaccent(btrim(regexp_replace($1, '\s+', ' ',...
1718:  SELECT lower(unaccent(btrim(regexp_replace($1, '\s+', ' ', 'g'), ' "')))

Here is the definition of that index in the old DB:

\d ix_semantic_mapping_normalize_title
  Index "public.ix_semantic_mapping_normalize_title"
       Column       | Type |        Definition
--------------------+------+---------------------------
 semantic_normalize | text | semantic_normalize(title)
btree, for table "public.semantic_mapping"

and the semantic_normalize function it uses:

\x \df+ semantic_normalize
Expanded display is on.
List of functions
-[ RECORD 1 ]-------+---------------------------------------------------------------------------
Schema              | public
Name                | semantic_normalize
Result data type    | text
Argument data types | title text
Type                | normal
Volatility          | immutable
Owner               | dm_admin
Security            | invoker
Access privileges   |
Language            | sql
Source code         |                                                                           +
                    |   SELECT lower(unaccent(btrim(regexp_replace($1, '\s+', ' ', 'g'), ' "')))+
                    |
Description         |

Any more information I can provide, to help troubleshoot this?

Cheers,

Gulli